Community Research and Development Information Service - CORDIS


POP Report Summary

Project ID: 676553
Funded under: H2020-EU.

Periodic Reporting for period 1 - POP (Performance Optimisation and Productivity)

Reporting period: 2015-10-01 to 2016-09-30

Summary of the context and overall objectives of the project

High performance Computing is becoming a fundamental tool for the progress of science and engineering and, as such, is becoming more and more important for increasing economic competitiveness. The growing complexity of parallel computers is leading to a situation where developers and users do not fully understand the underlying issues affecting the performance of their applications which often results in an inefficient use of infrastructure. Even when application developers are aware of the need to improve the performance and efficiency of their own code, they often do not have sufficient insight as to the root causes of their bottlenecks to be able to address them properly. This may lead to blind attempts to restructure code resulting in an overall lack of efficiency. The objective of POP (Performance Optimisation and Productivity) Centre of Excellence in Computing Applications is to help developers and subsequently businesses address their performance needs so that they can ultimately become more competitive POP is based on the premise that machines will become more and more complex as we approach exascale. The complexity, scale and variability in behavior for current HPC systems already exposes an important distinction between how developers and users think that applications and systems behave and how they actually behave.

POP offers a service to asses performance of a wide range of computing applications in a wide range of configurations from a few dozen processors to many thousands of processors. POP provides its Customers with detailed analysis of the issues affecting the performance of their code as well as suggests the most optimal way to alleviate these issues. POP targets application developers and users from all domains, including infrastructure operators, academic and industrial users. The estimated number of these applications in Europe is projected at 1500 codes. POP has the ambition of analyzing over 150 of these types of codes within the project lifetime.

However, POP does not stop at the service. POP is working to generate and expand the market. In our target customers (the communities of code developers, users and resource operators) POP aims to provide a recognized high value service that will save on developer effort and make the most of available resources while minimising cost. Based on the services from POP, application developers will make better-informed decisions to drive their work in the more productive direction. Infrastructure operators will ensure that their systems are provisioned in the most efficient way to maximise their Return on Investment. End-users (both academic and industrial) will stop wasting money and time by using applications operating below their optimal levels. The POP project also aims at establishing a link among those customers, promoting networking activities and ensuring that the POP users (customers) are offered the mechanisms to communicate among them as well as perform a key driving role in the activities of the Centre.

POP is one of the nine CoEs that was selected under the e-Infrastructure call (H2020-EINFRA-2014-2015/H2020-EINFRA-2015-1). POP started on 1 October 2015 and will run until 31 March 2018. The project has received EUR 4 million funding from the European Union‘s Horizon 2020 research and innovation programme.

For more information, visit the project website:

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

In the first months of the project, we focused on to defining and implementing the procedures required to manage the Center of Excellence. The organization of the Center of Excellence has been established and successfully validated tough the first year.

During the first twelve months, our main focus in the POP Community Development was on identifying and attracting potential POP users and maintaining and managing the community of actual and potential POP users. Initially all partners used their personal contacts to identify early users of the POP service. Additionally, we have advertised POP on a number of mailing lists, and identified potential customers by web searches, researching conference attendees, identifying software authors etc. However, our main focus has been on advertising POP at events where we can meet potential users face-to-face. In the first year we attended a total of 17 events. We presented POP at the majority of these events through one or more presentations or posters. Events have been identified because either they have a cross-sectoral focus on HPC (e.g. PRACE Days, International Supercomputing, TERATEC Forum), or they focus on an application area where HPC is heavily used (e.g. NAFEMS UK for CAE, Mobile World Congress for the mobile telecoms industry, a UK workshop on materials science etc.). This approach has given us good exposure to academic researchers and larger companies but it is harder to encounter SMEs at this sort of event. Up to now we have 185 potential leads contacted and 54 users that already requested a POP service. We note that so far 21% of our studies have been on code that is distributed commercially. Going forward we intend to continue with this general approach. However, we are also going to make a special effort to identify and contact ISVs, especially if they are SMEs.

We want users and potential users of POP to feel that they are part of a community rather than just customers of a service. To this end, we’ve launched a regular newsletter and have worked closely with WP7 to ensure a steady supply of content for the blog and social media. We have launched the POP Helpdesk to provide longer-term support for previous users who may have questions about services that they have previously received.

Additionally, we have begun to consider approaches to develop a strategy to ensure that POP is self- sustainable after the project ends, even though this work was not originally scheduled in the first project period. While we have yet to come to any definitive conclusions we have identified a number of possible models (e.g. subscription or consultancy model), each with their pros and cons. The choices will be set out in the next project period.

During the first year of POP, we received 70 service requests and by early November 2016 40 services have been completed. Most of them correspond to the basic performance Audit where the code is diagnosed with respect to the efficiency achieved (see Figure 1 below).

We are implementing the Proof-of-Concept (PoC) service activities to demonstrate to users how the optimization techniques suggested in WP4 can be applied. In particular, this includes complex code refactoring or applying advanced parallelisation techniques that can be performed on the source code or on kernels extracted from it or mock-up mini-applications.

We’ve created 5 questionnaires to collect the feedback on the services; one questionnaire for each of the three types of POP services (one for each type of service: Performance Audit, Performance Plan, and Proof-of-Concept), one questionnaire regarding Performance Tools, and one questionnaire to evaluate our suggestions for performance improvements with the aim or measuring global return on investment. We have sent the appropriate questionnaires to all customers who benefitted from a POP service as of the end of September 2016: 31 Perf romance Audit, 1 Performance Plan, and 3 Proof-of-Concept. Additionally, we’ve conducted phone interviews covering the six services provided by POP. So far 100% of our customers are either very satisfied or satisfied with the quality of the services.

Early customers were invited to participate in the Birds-of-a-Feather (BoF) session organized by POP during the ISC High Performance 2016 conference held in Frankfurt, on 22 June, 2016. Two of them gave testimony of their satisfaction with the POP services. This event attracted more than 40 users and potential users of the service.

Concerning training, two five-day workshops were organized (POP-EoCoE workshop at Jülich Supercomputing Centre, 8-12 December 2015, POP-EoCoE workshop at Maison de la Simulation, Saclay, 30 May – 2 June 2016) and the organization of two more are in progress to train users to use performance analysis tools themselves. The workshops contained presentations on usage of the performance-analysis-tools Score-P, Scalasca, Extrae, Paraver and Dimemas. Furthermore, the “bring-your-own-code” sessions allowed participants to start performance analysis of their applications within the workshop under the guidance of POP tools experts. At both workshops the number of participants was around 20, some participants were code developers and others were HPC experts working for the EoCoE project.

Finally, the communication and dissemination strategy has been developed. A corporate image for the POP project was created to ensure better visibility and recognition as well as a unique branding of the project. The project website is available under the URL In addition, an online blog for the project was established that features articles about major project results, customer success stories, reports from training and dissemination events and other news items interesting to POP customers. Traditional social media channels like LinkedIn and Twitter, and the “News” section on the POP website are linked with the blog. Starting in June 2016, a quarterly (email) newsletter is mailed to a list of subscribers. Initial marketing material was created at the beginning of the project for POP community development. This includes a service catalogue in the form of a leaflet, a contact (business) card, as well as a PowerPoint presentation that can be distributed and presented to potential customers and users at communication events or during customer’s visits.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The first results of the POP performance analysis services are promising. All audits have provided useful suggestions for performance improvements to the clients. The three closed PoCs resulted in improvements in production runs, in a near ideal speedup of the concerned parts of the code and in much better scalability. So far all POP customers were very satisfied or satisfied with the provided services according to the satisfaction questionnaires and interviews.

POP identified 185 potential users, received 70 requests from 54 users in the first year of the project and completed 40 services by early November 2016.

Short-term project impact concerning increased code performance in terms of speed and quality or better return on investments (ROI) can be measured at a later stage of the project once modified codes were running in production for a given period of time.

The long term potential impacts of the project are listed in Table 1 below.

Related information

Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top