When Strathclyde Pension Fund decided to review its global custody service provider last year, we were anxious to be as thorough as possible in gathering information about would-be custodians, to reduce subjectivity as far as possible and to diminish the overall impact of pricing in the evaluation process.

The first step was the development of the request for proposal (RFP). Ultimately this had 17 main sections, each representing an important criterion. The RFP finally comprised over 100 questions, some of which had multiple sub-sets to them.

In addition, there was the inevitable section on pricing which had to be completed against a standard data set provided by the fund, together with a general section which called for three 500-word articles on particular topics designed to draw out the prospective providers' views on the current state of the custody industry, the risks in custodian selection and operational risks in international markets.

The approach adopted in this general section features in most RFPs issued by the fund, albeit tailored to the brief being researched. This favourite tool is however notoriously difficult to evaluate on a comparative basis and is known locally as 'the tie-breaker'.

Prior to the RFPs being issued the 17 main sections were ranked in descending order of importance using a system developed by the Chartered Institute of Public Finance and Accountancy . Essentially this involves comparing each criteria against the others and scoring them as being of higher importance (2), equal importance (1 ) and lower importance (O). The results were then summed and the rankings shown in the table established.

The 12 completed RFPs were initially screened and ranked for price. The three most expensive were eliminated without further evaluation (their prices ranged from 189% to 230% of the cheapest).

The next step was the detailed evaluation of the RFP which was intended to reduce the remaining prospective custodians to four who would be interviewed and visited in their office. The evaluation team (three people working independently) were instructed to mark each criterion in each RFP on the basis of 1 (grossly unsatisfactory) to 5 (very good).

Having established base scores for each criterion, each evaluator then weighted the scores according to a predetermined formula that assigned a factor of 3x to the top six 'key criteria', 2x to the following six 'important service criteria' and 1x to the remaining 'additional service criteria'.

This enabled the evaluators to come together to compare scores, discuss points of detail or differences and adjust where necessary to agree on the final four. In an ideal world the evaluators would agree perfectly but this was not the case. One custodian (the ultimate winner) was ranked first by all three evaluators while two of the final four were ranked in the top four by two of the three. This left five submissions ranked in the top four by only one evaluator and I have to record that the chairman of the evaluation panel imposed his will in respect of the custodian to fill this final place.

Was it a success? Partially would have to be the answer. It was an honest attempt to set price to one side for a time and to concentrate as objectively as possible on quality. To some extent it succeeded. The final choice was a custodian which everyone had ranked first in evaluation and which stood up well both at interview and on the site visit. However, the extent of disagreement among the evaluation panel in selecting the final four was disappointing.

And price? Yes, you guessed - the final choice happened to be the firm that was seen to be cheapest when the envelopes were first opened. Perhaps a lot of hard work could have been saved after all !

Geoff Singleton is depute director of finance of Glasgow City Council

Topics