PensionsEurope has urged European consumer finance campaign group Better Finance to make changes to the research behind a report on the real returns of pension savings of Europe, arguing that it is otherwise “comparing apples and pears”.

It follows the fourth edition of a Better Finance report that aims to show the real returns of pension savings – after charges, inflation, and taxes – in 15 European countries.

The 2016 report, published in September argued that once inflation, charges and (where possible) taxes were taken into account, private pension products have often performed poorly.

“Unfortunately our research findings show that most pension savings did not, on average, return anything close to those of capital markets, and in too many cases even destroyed the real value for European pension savers (i.e. provided a negative return after inflation),” said Better Finance.

However, PensionsEurope criticised the report in a paper published last week, which included hard-hitting comments from its member associations in Bulgaria, Germany, Italy, and Spain. PensionsEurope said these organisations had found “substantial discrepancies and wrong interpretations”.

For example, the Spanish Association of Collective Investment Schemes and Pension Funds (Inverco) said that although it appreciated Better Finance’s research effort and the difficulties involved, it had “still found a number of inconsistencies and mistakes offering a distorted vision of the Spanish pension funds industry”.

It said its main concern was the “deliberate omission of tax impact in Spain when comparing Spanish pension funds with those from other [EU] member states”.

The European occupational pension fund trade body was at pains to strike a constructive tone, however. It suggested areas of improvement, and welcomed “the research on the quality of occupational and personal pensions and the outcome of pension savings”.

Janwillem Bouma, chair of PensionsEurope, said: “In [our] paper, we highlight numerous specificities that the research should take into account in order to give a realistic picture of the quality and outcome of pension savings. If ignoring these specificities, the research faces a serious challenge of comparing apples and pears.”

Matti Leppälä, secretary general of the association, offered PensionsEurope’s help to improve the methodology of Better Finance’s report: “Particularly, we invite Better Finance to use the data and time periods which are consistent and comparable, focus on both the accumulation and payout phase, and explore the benefits in addition to the costs.”

Better Finance today said that it welcomed the offer of cooperation, but that it also wanted “to clarify some misunderstandings as to the objectives, the methodology, and scope of the research effort”.

“Our research goal is far more specific and far less ambitious than the one Pensions Europe highlights in its headlines: the aim is not to analyse ‘the quality and outcome of pension savings’, but to improve the transparency on the real returns of pension savings in Europe,” it said.

It called on trade associations to help plug a data gap and improve the transparency of data on pension returns, “since the bulk of the criticism aimed at the report by Better Finance originates from the lack of available data on returns”.

Both organisations have noted that the European Commission wants the European financial supervisory authorities to carry out out work on the transparency of long-term retail and pension products, including an analysis of actual net performance and fees.

Speaking at the launch event for Better Finance’s pension savings report in September, Commission vice-president Valdis Dombrovskis said that returns on retail investment products and pensions can be heavily influenced by fees levied by asset managers and intermediaries and that the “the right data on the net performance and fees of the most commonly sold products is important […] to ensure more competition and better service for consumers”.

IPE understands that EIOPA – the supervisory authority responsible for the insurance and occupational pensions sector – has yet to be formally assigned such work by the commission.