Having submitted a number of proposals in response to the first calls of Horizon 2020, we have started receiving feedback from the European Commission in the second half of August for those proposals submitted in early spring. The Commission (or its Agencies) evaluates the Horizon 2020 proposals with the help of independent experts. Once the evaluation sessions are completed, the applicants receive feedback from the Commission in the form a of so-called Evaluation Summary Report (ESR), which provides detailed and specific remarks (positive and negative) based on different evaluation criteria and in a systematic way following the specific sections of the submitted proposals.
We have assessed and summarised the main conclusions of a number of ESRs that we have received, and we are providing via this blog post some highlights and recommendations for proposers. One notable change in the ESRs compared to FP7 is that the evaluators now provide specific remarks for each sub-criteria within the main evaluation criteria (typically Excellence, Impact and Implementation), and this is structured and presented accordingly in several sub-sections in the ESR. The remarks are quite elaborate and specific, giving an impression that the Commission and evaluators have actually managed to conduct a detailed assessment of the proposals despite the rather high number of proposals that were received.
Lessons learned and recommendations for proposers for future proposals (Evaluators’ remarks excerpted from ESRs shown in brackets in italics):
- The evaluators actually pay attention to the project design, such as how the overall and specific objectives of the project are formulated. ((...) But the overall objective that is connecting all the individual objectives is not clearly stated and therefore it weakens the focus of the proposal)
- It is always good to distribute tasks among the partners, especially the drafting of the WP descriptions; however, ensure that someone (e.g. the Coordinator) oversees this process and revises all this input in order to harmonise them and ensure logical linkages between the WPs. Otherwise the evaluators might get frustrated trying to figure out the connections between WPs and make sense of the resource distributions among WPs. ((...) in WP3 tasks are not fully justified; in WP5 explanations are confusing. The connection between WP5 and WP 6 is not fully clear)
- Avoid generic texts and formulas – e.g. make business plans specific to the project concept and its outputs. (Another minus is the business plan, which is presented in a theoretical way and is not specific)
- Even in Innovation Actions where you might want to keep project outputs confidential due to IPR considerations, try and foresee at least 2-3 public deliverables. (The most important shortcoming is that, basically only one report is planned to become public and that is the final project report)
- It is a good approach to build on the results of past projects and to demonstrate that your consortium has direct access to them. (The project makes use of previous knowledge coming from other EU funded projects that were carried out with the participation of some members of the proposing consortium.)
- Makes sure to allocate adequate staff effort for dissemination and exploitation activities (especially in case of projects delivering novel solutions) and provide some staff effort in the Project Management work package to the partners to allow them at least to take care of project reporting duties. (However the effort in WP7 - Dissemination and exploitation, is not duly justified (5.6% of total person/months), in comparison with the proposal potential. It is also not clear why the effort in WP8 - Project Management, is only attributed to the project coordinator, with no effort allocated to any other partner)
- It is very important to identify and define the end-user(s) (final beneficiary(ies)) of your research and innovation action. This must be clearly and explicitly stated with due highlight of the exploitation potential and plans. (The proposal does not clearly explain who will be the final user of the solution)
- Providing quantitative indicators is good; however, make sure that this is worked out realistically and based on a sound methodology. (Although there is a quantification of indicators, the rationale is not fully credible and easy to understand. For example 70% energy consumption reduction is not explained in detail + (…) the estimation that the project will give rise to 400.000 new jobs is not convincingly demonstrated)
- Forward/strategic thinking and a market-based approach are considered positively. (The proposal approach is credible and the topics are achievable, as it includes all the needed steps to produce useful results for the market)
- Operational capacity of the institutions and their reputation in the relevant sector/domain are indeed taken into account by the evaluators. (A good point is that the leader in the sector, XYZ, is the member of the consortium)
We are now going through Grant Agreement preparations for a number of our proposals which have been positively evaluated. Therefore, once completed, we hope to provide some useful insight and tips also on this process in the coming weeks.
Ömer
