Programme performance overview - Introduction and Methodology

Introduction

The Commission is committed to presenting accurate, reliable and understandable performance information relating to EU spending programmes. This annex to the Annual Management and Performance Report contains the programme performance statements, which are also working document 1 of the 2024 draft budget. This overview presents:

• information on the financing of initiatives relating to climate, biodiversity, gender equality, and sustainable development goals (SDGs) objectives, as provided for in point 16(d–g) of the interinstitutional agreement. Information on the contribution of the EU budget to the European Commission’s priority of promoting the digital transition is also provided.
• all of the EU spending programmes for the 2021-2027 period in a concise and uniform format. In addition, it presents the implementation and the performance of those programmes for the 2014-2020 period for which relevant payments were still being implemented in 2022.

For each programme, the overview contains a file with the following information.

• Concrete examples of achievements of both the 2021-2027 and 2014-2020 programmes.
• For each 2021-2027 programme:
  • the 2021-2027 budget;
  • the rationale and design of the programme;
  • the budget implementation;
  • the contribution to horizontal priorities (green budgeting, gender and digital);
  • an assessment of the performance of the programme based on currently available information.
• For each 2014-2020 predecessor programme (if applicable):
  • the budget implementation;
  • an assessment of the performance of the programme based on currently available information.
• Contribution to SDGs.

The ‘Performance assessment’ section presents a summary of the overall progress towards achieving the programme objectives. In addition, the ‘Key performance indicators’ section includes an assessment of whether the indicators are on track to reach their targets.

It should be kept in mind that, by the end of 2022 many 2021-2027 programmes are just starting to be implemented, in those cases implementation data is still limited. Moreover, given the multiannual life cycle of EU spending programmes, information on actual results may only become available a number of years after the start of programme implementation. This applies, in particular, to the European Structural and Investment Funds and other long-term investment programmes. On the other hand, the implementation of certain programmes from the 2014-2020 programming period is ongoing, and will continue for a number of years, as provided for in their legal bases. In reporting exercise we also report on the results of such ongoing 2014-2020 programmes.

 

Financial information: methodology

The purpose of the financial information presented in the programme performance overview is to enable the reader to make the link between the funds made available for a specific topic and the results actually achieved through the use of these funds.

The methodology takes into account the fact that the EU budget uses different categories of expenditure (or ‘fund sources’), and it is important to include all of them to present a full picture of the financial efforts made to achieve the objectives. Each programme website presents financial information on both the 2021-2027 programme and its predecessor programme(s), as long as they remain relevant. Due to the evolving nature of the EU budget, additional fund sources have been included in the financial information on the 2021-2027 programme. For this reason, the methodology varies between the two multiannual financial frameworks.

With respect to the 2014-2020 programmes, the following should be noted:

• Budget: The total budget of a programme principally includes the commitment appropriations authorised on an annual basis by the European Parliament and the Council, i.e. the voted budget.
• Implementation: In addition to the above, this year’s report contains the implementation made on the basis of two additional sources, as follows:
  • Expenditure relating to amounts carried over from the previous year.
  • Payments made on the basis of recoveries and repaid advances for the programmes that the Commission implements in cooperation with the Member States: the European Regional Development Fund, the Cohesion Fund, the European Social Fund, the Fund for European Aid to the Most Deprived, the European Globalisation Adjustment Fund, the European Agricultural Guarantee Fund, the European Agricultural Fund for Rural Development, the Asylum, Migration and Integration Fund, the Internal Security Fund and the European Maritime and Fisheries Fund.
    Note: Payments made in the 2014-2020 period on the outstanding commitments from the 2007-2013 period are not included in the calculations so as to focus only on the implementation of the 2014‑2020 programmes and the related achievements.
• Any exceptions to the above methodology are specified in footnotes in the programme websites concerned.

With respect to the 2021-2027 programmes, the following should be noted.

• Budget: The total budget includes commitment appropriations arising:
  • from the voted budget, i.e. authorised on an annual basis by the European Parliament and the Council;
  • from the voted amounts of fines pursuant to Article 5 of the financial regulation (this source in only used for 2022, 2023 and 2024 only).
  • from NextGenerationEU;
  • from decommitments made available again for research programmes, pursuant to Article 15(3) of the financial regulation (for the years reported on only, i.e. 2021 and 2022); and
  • from assigned revenue resulting from the participation of European Free Trade Association states, participation of PECO states, and other countries and entities in certain EU programmes (for the years reported on only, i.e. 2021 and 2022)
• Implementation: In addition to the above, this year’s report contains the implementation made on the basis of two additional sources, as follows:
  • Expenditure relating to amounts carried over from the previous year.
  • Payments made on the basis of recoveries and repaid advances for the programmes that the Commission implements in cooperation with the Member States: the European Regional Development Fund, the Cohesion Fund, the European Social Fund, the Fund for European Aid to the Most Deprived, the European Globalisation Adjustment Fund, the European Agricultural Guarantee Fund, the European Agricultural Fund for Rural Development, the Asylum, Migration and Integration Fund, the Internal Security Fund and the European Maritime and Fisheries Fund.
    Note: Payments made in the 2021-2027 period on the outstanding commitments from the 2014-2020 period are not included in the calculations, in accordance with the approach used for the 2014‑2020 programmes (see above).
• Some programmes (e.g., ESF+) can pursue their objectives by making financial contributions to other programmes, which then use them to fund their own activities. In such cases, these transferred amounts are included within the financial information of the recipient programme(s) and excluded from the financial information of the contributing programme. This convention ensures no double-counting of resources.
• Any exceptions to the above methodology are specified in footnotes in the programme websites concerned.

 

Key performance indicators: methodology

SELECTION OF INDICATORS

The performance framework for the 2021-2027 EU spending programmes includes more than 800 indicators measuring performance against more than 170 specific objectives. These indicators are included in the respective legal bases, typically following a proposal by the Commission and after negotiations between the co-legislators.

The programme performance statements presents a selection of the most relevant core performance indicators. For the 2021-2027 spending programmes, the selection of indicators was made based on the following criteria, aiming to present a balanced and representative picture of programme performance:

• a maximum of nine indicators per programme;
• coverage of specific objectives;
• a focus on choosing the most representative indicator for the specific objective;
• a focus on output and result indicators, rather than on impact indicators, in the light of the early stage of implementation and of the expected lag with which impact indicators will show effects;
• indicators with targets (or indicators for which targets will be defined), in order to estimate the progress towards the target.
• a preference for indicators that can be understood by non-specialist readers.

In the case of the 2014-2020 spending programmes, the chosen indicators are those that were already included in the programme performance overview from last year and on which reporting has continued this year. This year’s programme performance statements have continued to report on 2014‑2020 indicators for programmes with relevant payments still being implemented, except for indicators that:

• have been discontinued;
• show a very limited change in results compared to the previous year (one such circumstance is when the target had already been reached in a previous year);
• provide limited value in assessing the performance of the 2014-2020 programmes at this stage of the life cycle (e.g. input and output indicators for some programmes are more relevant during the first years of implementation, while result and impact indicators are more relevant at a later stage).

It is crucial to bear in mind that the information contained in the indicators can only provide an indication of the overall performance and achievements of each specific programme. It is only possible to make comprehensive statements about the ultimate performance of programmes by taking into account the specific implementation context, including qualitative as well as quantitative elements. The Commission does this in the context of regular evaluations of its spending programmes.

 

DEFINITION OF TARGETS

Targets are defined at the beginning of the programme implementation period and come in various forms (e.g. quotas, benchmarks, numerical goals). The methodology used to set the target for each indicator can be found in the indicator metadata Excel file.

In some cases, the final target is set for 2027, the end of the programming period. However, account should be taken of the specific nature of the shared management programmes, which are characterised by a long start-up phase (e.g., planning, programming, project selection, authorisations) followed by a long implementation cycle. As such, the appropriations for the European Structural and Investment Funds can still be implemented in the 3 years following the commitment of the funding (the so-called ‘n + 3’ rule), therefore the final target is typically set for 2030.

 

DEFINITION OF BASELINES

A baseline is a measurement taken prior to a specific intervention, which allows the results before and after (i.e., with and without) the intervention to be compared. In the context of the EU budget, the baseline is the measurement of the indicator before the start of EU budget funding for the current programming period. The methodology used to estimate the baseline for each indicator can be found in the indicator metadata Excel file.

Because of its nature, a baseline is not always available for all the indicators, whereas in other cases the baseline should be considered to be zero. This is the case, for example, for the ‘output indicators’ relating to the specific deliverables of the intervention, such as the number of projects funded by the EU budget.

It is important to make a distinction between a baseline and a historical benchmark or reference. The measurement of an indicator before the 2021-2027 period should not automatically be considered as a baseline, since its measurement could have been influenced by the EU budget actions from the previous period (2014-2020).

The baseline of an indicator is used when the indicator meets the following three conditions.

• A quantitative baseline is available in the programme statements.
• The baseline meets the definition of measurement before EU budget intervention.
• The target represents an improvement (i.e., a better result) compared to the baseline – otherwise the progress could not be calculated. Depending on the indicator, an improvement can be an increase, or a decrease compared to the baseline.

 

DEFINITION OF ‘PROGRESS’

The ‘progress’ of an indicator provides a consistent presentation across programmes of the evolution of the indicator from the baseline (if applicable) to the target. In the programme performance overview website(s) the progress is shown using bar charts for the selected indicators.

 

General formula

Progress % = (last year result – baseline) / (target – baseline)

 

Example

Programme: Implementation and exploration of European satellite navigation systems (Galileo and EGNOS)

2021 progress % = (28 – 4) / (30 – 4) = 92%

 

Adaptations

Specific adaptions, if any, are noted in the footnotes at the bottom of the respective figures. Examples of such adaptations are as follows.

1. In cases where the target and results are annual values:

   Progress % = number of years with results above the target (or annual milestones) / total number of years

2. In cases where the progress should be expressed in a cumulative way from the beginning of the 2014-2020 period, and the annual results are not cumulative:

   Progress % = (sum (annual results) – baseline) / (target – baseline)

3. In cases where the progress cannot be expressed in a cumulative way, and the ‘last year result’ does not reflect the progress of the programme during the period as a whole, then the ‘last year result’ is replaced in the formula by the ‘average of the annual results from the beginning of the period’:

   Progress % = (average (annual results) – baseline) / (target – baseline)

In cases where annual milestones are available and the progress of the programme is better reflected comparing the results to the annual milestones, then targets are replaced in the formula by annual milestones.

 

INDICATOR ASSESSMENT

The results of the indicators are assessed using the following definitions.

• ‘Achieved’: if the indicator has achieved at least 95% of the target and there is no possibility that the achievement could be reversed/jeopardised before the target year.
• ‘On track’: if the indicator is expected to reach at least 95% of the target based on the evolution of the results and the ongoing actions.
• ‘Moderate progress’: if the indicator shows a positive evolution but the results so far do not allow a conclusion to be drawn on whether the indicator is on track to reach its target.
• ‘Deserves attention’: if there is a risk that the indicator will not reach the target unless significant changes are implemented.
• ‘No data’: the data available is not enough to assess the progress of the indicator towards the target, for example there is no target or no recent results.

 

CUT-OFF DATE FOR PERFORMANCE INFORMATION

The most recent available performance information is used. For those programmes that are directly managed by the Commission, this mostly concerns reported achievements measured at the end of 2022. The programmes under shared management present values recorded and reported by Member States of the situation as at the end of 2021. The programmes under indirect management present a mixed picture: some have achievements reported up to 2022, while others depend on data sources provided by the international organisations that implement the actions (e.g. the United Nations family), and may therefore be reported with a lag.

 

Download the introduction and the methodology