To say that pension funds exist to convert retirement savings into pension payments tells only half the story. The other half of the story must explain how well or how badly they do so. In other words, the organisational performance of pension funds is an important issue. This raises three questions:
q By what criteria should we judge the organisational performance of a pension fund?
q Using those criteria, how well is the average pension fund actually performing?
q What seem to be the critical factors that affect organisational performance?
By addressing these questions, we can set standards for measuring pension fund performance, and for developing strategies to improve it.
Cost Effectiveness Measurement (CEM) began measuring the organisational performance of US and Canadian pension plans in 1991. The process commenced with the investment side of the plans. A comparable benchmarking process on the pension administration side was launched last year.
The proper evaluation of the investment performance of a pension fund requires an appreciation of four key concepts:
q There are two basic ways a pension investment programme can add ‘value’ to a pension balance sheet: first, by undertaking asset mix policy risk, and, second, by undertaking active management risk and costs.
q This implies that we need to measure both asset mix policy risk and the ‘value’ it is producing, and active management risk and cost, and the ‘value’ they are producing.
q Policy risk and its ‘value’ can be assessed by observing the relative return behaviour of a risk-free portfolio that mirrors the pension liabilities versus a portfolio that captures the performance of the chosen asset mix policy, passively implemented.
q Active management risk and cost, and the ‘value’ they are producing, can be assessed by observing the relative return behaviour of the actual pension fund portfolio, net of active management costs, versus a portfolio that captures the performance of the chosen asset mix policy, passively implemented.
Let us see what happens when we apply these principles.
Tables 1 and 2 set out the actual 1994–97 experience for the 80 US and 56 Canadian pension funds in the CEM database with four years of continuous experience. Table 1 deals with the asset mix policy results, Table 2 with the policy implementation results.
Clearly, taking on asset mix policy risk was extremely well rewarded, producing average policy value added/policy risk ratios of 1.2 in the US, and 0.9 in Canada. Expressed differently, after applying reasonable risk penalties to the respective policy value added, there is plenty of positive risk-adjusted net policy value added (Policy RANVA) left over.
In both the US and Canada, the average pension fund realised zero payback on the incremental costs and risks spent on active implementation strategies. Thus on an after-cost basis, the net implementation value added/implementation risk ratios were –0.2 in both countries. Expressed differently, after applying reasonable risk penalties to the respective net implementation value addeds, the average funds lost –0.7% and –0.6% of implementation RANVA per annum respectively.
A major study was conducted in 1997 to see whether it was possible to identify the key factors influencing the production of positive implementation RANVA. Three statistically significant factors were identified:
q Fund size: Larger funds were more likely to produce positive implementation RANVA than smaller ones.
q Proportion passively managed: Funds with higher proportions of assets passively managed were more likely to produce positive implementation RANVA than funds with lower proportions.
q Organisation design quality: Funds with higher organisation design quality scores were more likely to produce positive implementation RANVAs than funds with low scores.
The broad implication of these findings are that pension funds can indeed take concrete steps to improve organisational performance.
CEM began to work with Dutch pension funds in 1998. Tables 3 and 4 include the average results for the four Dutch funds for which we now have four years of continuous data. The re-sults should be treated with caution, as a number of data and measurement issues have yet to be finalised. Nevertheless, the preliminary results lead to some interesting findings, which are unlikely to change with data revisions.
The average Dutch fund took about half the policy risk which its North American counterpart took over the 1994–97 period, and thus earned a considerably lower RANVA. Note however, that the Dutch funds faced almost the same favourable policy reward/risk ratio as their North American counterparts.
Table 4 suggests that the four Dutch funds had better average policy implementation experience than the average US and Canadian fund in the CEM database over the 1994–97 period. Average gross implementation value added was higher, and unit operating costs were lower. It appears the Dutch funds took on somewhat higher levels of implementation risk than the North American funds. However, they get an adequate payback. As a result, the implementation RANVA averaged a positive 0.4% a year.
To be useful, the measurement of organisational performance must measure the right things in a standardised manner. In the case of pension funds, this means measuring liability returns, policy returns, actual returns, policy risks, implementation risks and costs in a standardised manner.
When that is done, much can be learned. Look at Figure 1, for example. It plots net implementation value added versus implementation risk at the total pension fund level. Even though Figure 1 contains results for US, Canadian and Dutch funds, the plots are perfectly comparable.
Clearly, the average fund did not get a positive payback on its implementation cost and risk over this period. There are many more funds below the zero net value added line than above. Further, there is no correlation between implementation value and implementation risk.
Figure I also places the Dutch z-score performance test in context. Four observations might be:
q Without careful, standardised measurement of actual returns, policy returns and operating costs, the z-scores will be very ‘fuzzy’ numbers.
q Based on the actual experience of 140 pension funds (including the four Dutch), drawing a ‘failure’ line at a net value lost of –2% a year, gives you about a 10% failure rate.
q In fact, most of these 140 funds would have served their stakeholders better if they had not incurred any additional implementation risk and cost at all.
q Our research shows that the previous observation is especially true for smaller funds which are managed 100% actively, and which have not paid careful attention to how they are organised.
Keith Ambachtsheer is head of KPA Advisory Services, in Toronto. This article summarises an address made at a conference sponsored by the Nederlands Studie Centrum titled “Kosten en Rendementen van Pensioenfondsbeheerd-ers” in December 1998. Further details: www.kpa-advisory.com

Topics