Skip to content

2.3.4. Quantified objectives and quantification criter

Peter Mahlmann edited this page Sep 26, 2016 · 1 revision

(Recommended length: 2 pages) Consider the expected project results (cf. §2.3.3), and for each one of them define appropriate quantification criteria (Key Performance Indicators - KPIs) that will be used to measure the objective achievements, i.e. what will enable the consortium and evaluators to measure during the course of the project the progress achieved towards the goals. The KPIs should not cover the steering & management of the project, but cover actual exploitation oriented project results. Example 1: for the detection of ships on coastal borders, the ship detection and recognition rates as well as the required processing time could be considered, and confronted with a target defined by the end users (e.g. 90% detection rate, not more than ten false alarms per hour on the typical traffic of a given area, and at least 5 images per second analysed in a continuous stream with a single workstation). Example 2: for an HPC framework applied on quantum physics simulation, considered KPIs can include the coverage of accelerated code (how much code is now “HPC-ready”), the processing time reduction (normalised by the hardware cost and/or by the power consumption), the impact on the implementation time (for experienced as well as for new developers), and/or the performance gain for the simulation tool itself, from a user point of view (e.g. latency between request and results, real-time visualisation, etc.). Example 3: for a standardised model-based framework for the transportation industry, KPIs can cover the percentage of models that can be simulated (on predefined industrial use-cases with existing code), the performance impact (e.g. with an objective of having that impact below 1%), the requirements and specifications coverage of the current implementation, the number of active members in the open source community and the number of industrial end-users that have adopted the framework (or that are at least experimenting with it). The KPIs can also include a set of binary goals to be achieved (e.g. full UML integration, real-time debugging and on-the-fly code recompilation within a simulation, etc.). This subsection should convince reviewers that the clear analysis and quantification of project progress will be possible during the project lifetime.