Organising dissemination on Results of projects on Chemical Evaluation, Spreading Techniques for Risk Assessment

Informazioni relative al progetto

ORCHESTRA

ID dell’accordo di sovvenzione: 226521

Progetto chiuso

Data di avvio 1 Settembre 2009

Data di completamento 31 Agosto 2012

Finanziato da

• Specific Programme "Cooperation": Environment (including Climate Change)

Costo totale € 1 048 762,92

Contributo UE € 910 929,70

Coordinato da

ISTITUTO DI RICERCHE FARMACOLOGICHE MARIO NEGRI
Italy

Final Report Summary - ORCHESTRA (Organising dissemination on Results of projects on Chemical Evaluation, Spreading Techniques for Risk Assessment)

Executive Summary:
European chemical-related regulations aim at secure the safety of chemicals for human beings and the environment. Thus, chemical substances circulating in Europe need data on several properties, such as toxicity.
The classical way to assess toxicity of chemicals has been to use animal tests (also called in vivo tests). However, in the last years, the European Commission has started to favour and explicitly promote the adoption of so called “alternative methods” within the issued regulations. These approaches alternative to animal testing include computer based virtual assessment of chemicals (also called in silico methods).
Since the adoption of these new regulations the European Commission has started to fund collaborative in silico-related projects. The aims of these projects is to improve understanding of these methods, to study how they can be integrated with other approaches, to develop new models, to promote their use, etc.
ORCHESTRA (Organising dissemination on Results of projects on Chemical Evaluation, Spreading Techniques for Risk Assessment) was a European project funded with the aim of studying and improving the dissemination of in silico methods for toxicology. Starting from a number of EC projects as case studies, ORCHESTRA analysed both their outcomes and the dissemination strategies adopted. Moreover, a database of stakeholders possibly interested in in silico methods was compiled, which included people from chemical industries, regulators and consultants as well as academics and members of governmental and non-governmental organizations.
The identified stakeholders were involved, at different levels and timings, during the whole duration of the project. The relations were not unilateral, stakeholders were asked several times to actively participate. Questionnaires, surveys and exercise about in silico methods were developed and sent to them, with the aim to better understand their view of in silico methods. Moreover, tens seminars and other events were organized and held directly at regulators’ and industries’ offices. Group studies (both live and on line) were also organised with stakeholders.
ORCHESTRA actively participated to several important international events related to chemistry, toxicology and in silico methods (e.g. SETAC and QSAR2012). Members of the project gave presentations and lessons during these events, results from ORCHESTRA as well as from other in silico-related projects were presented.
Dissemination and informative material about in silico were produced, including newsletters, brochures, and DVDs. The material produced, as well as more details about these methods and the legislative framework in which they are inserted, are present in the ORCHESTRA official web portal (http://www.orchestra-qsar.eu/).
More than ten movies have been produced and made available on the ORCHESTRA website, with interviews to regulators, industry representatives and scientists, including also lessons on in silico methods.
Another important goal was the compilation of the ORCHESTRA e-book, entitled "theory, guidance and applications on QSAR and REACH”.
Finally, the on line course “QSAR methods for REACH” about theory, applications and projects related to in silico methods was made available on the ORCHESTRA website.
Project Context and Objectives:
An effective dissemination requires on-going investigation of the significance of the research for potential users and stakeholders, and of the issues around its use. This summary of the context is therefore the result of work on the project as well as a precursor.


The REACH regulation

It is currently estimated that there are around 30,000 chemicals in use in the EU that have never been subject to approved testing. Regulatory concern to protect human health and the environment means that the situation is now changing.
June 1st 2007 has been a key date in the context of chemical substances circulation within Europe; it has seen the REACH regulation entering into force. REACH is the acronym of “Registration, Evaluation, Authorisation and Restriction of Chemicals” and is the common name of the REGULATION (EC) No 1907/2006 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 18 December 2006. This legislation requires that existing chemicals now be formally evaluated and registered. In recognition of the huge scale of this task, the European Chemicals Agency (ECHA), describes REACH as “the World’s most ambitious chemicals legislation”.
As stated in the homepage of REACH official webpage :

“The main aims of REACH are to ensure a high level of protection of human health and the environment from the risks that can be posed by chemicals, the promotion of alternative test methods, the free circulation of substances on the internal market and enhancing competitiveness and innovation.”

It is therefore clear that REACH aims at protecting human health and the environment, by means of innovative methods for assessing the possible risk of chemicals, alternative to classical animal (in vivo) testing. REACH applies to every chemical (above one tonnage/year) circulating within the European territory. Using in vivo approaches to fulfil all the requested tests would pose three main problems:
1) It would need a huge amount of animals (creating disappointment within the public opinion),
2) It would cost too much money to the chemical industries, especially for Small and Medium Enterprises (SMEs) and
3)It would take too much time.

Under REACH it is the responsibility of industry to generate the necessary evidence of safety/toxicity, to explain how the risks are managed in practice and to submit the documentation to regulators. Companies, who manufacture or import everything from cleaning products and clothes to electrical appliances, have to register every chemical substance. The European Chemical Industry Council (CEFIC) describes the implementation of REACH as “one of the most challenging tasks the chemical industry has ever faced”.
To make large scale evaluation achievable, REACH requires that companies which use the same chemical (i) share their existing data, (ii) collaborate on further evaluation and registration and (iii) use alternative methods where possible. Officially, innovation is encouraged and animal tests must be used only as ‘a last resort’.
While these three demands offer a potential route to making it possible to evaluate tens of thousands of chemicals, they also demand some radical changes in industry practices. In addition to the potential complexities of collaboration, the requirement to share existing data includes previously confidential data. REACH also encourages a shift away from the use of a single animal test as evidence, towards using a ‘weight of evidence’ approach in which different methods produce complementary ‘lines of argument’. In this way, the use of alternative methods takes place in the context of changing practical and intellectual demands in the science of regulatory toxicology. The technical use of a new method is only one part of the challenge.
The current period, 2008-2018 is a historic moment when the REACH legislation affects companies across the EU economy. The most toxic chemicals, and those traded in the largest quantities across the EU, had to be evaluated and registered by December 2010. Chemicals traded in smaller quantities have to be evaluated and registered before May 2013 and before 2018. These deadlines progressively apply to chemicals traded in smaller quantities, include larger numbers of chemicals and so impact on a wider range of companies, including SMEs.
For individual companies, the demands and the deadlines are critical: if they don’t evaluate and document the safety and toxicity of a chemical satisfactorily by the deadline, they will no longer be able to produce, import or sell any products with that chemical in the EU. (The parallel regulatory demands for biocides led to a dramatic reduction in the number of biocides on the market.)


Computer-based methods to comply with REACH requests

Among alternative methods, REACH explicitly cites computer-based (in silico) models, such as those developed using Structure-Activity Relationship (SAR) and Quantitative Structure-Activity Relationship (QSAR) approaches.
The hypothesis on which (Q)SARs are based, is that the properties (e.g. toxicity) of chemicals are related to their structure. Thus, if it possible to represent the chemical structure using numerical values (called molecular descriptors), it is also possible to correlate these values with a property of interest.
(Q)SARs present many advantages which can help facing the issues of time, costs and animal consuming (thus fulfilling the 3Rs principle):
1) These approaches use currently available experimental data to build predictive models. These models can then be used to assess chemical with no experimental data, avoiding or at least reducing the use of animals.
2) They are faster than both in vitro and in vivo experiments, allowing to screen large dataset of chemicals in minutes or, at most, in few hours. The time needed depends on the number of chemicals and the complexity of the model used.
3) They are cheaper than the other two approaches. Free predictive software have been developed by EC funded projects (e.g. CAESAR and DEMETRA) and national agencies (e.g. US EPA). Commercial software have been also developed for years. Even if their licence fees are in some cases very expensive, the cost is always much lower than in vivo and in vitro experiments.

Moreover, in silico methods have several advantages:
- For regulators: as already seen, in silico methods are able to process even thousands of compounds within hours. Thus, they are suitable to solve the urgent problems of the assessors, which have to face the problem of the evaluation of a huge number of compounds. For instance, tens of thousands of chemicals have to be evaluated for the recent legislation called REACH.
- For environmental and health protection: the availability of fast tools capable to process a large number of compounds can improve the knowledge on the environmental impact of contaminants, including their related compounds (transformation products and impurities of industrial compounds). Thus, the availability of these computer tools will improve the knowledge on all pollutants and contaminants, and increase the environmental and health protection.
- For industry: since in silico methods are cheaper (or even free) than, they can help industries, which bear the increased costs related to the deeper environmental and toxicological evaluation currently requested. In this way, even millions of chemicals can be processed, and only the safer ones can be planned for further development.
- For animal protection: in silico methods do not use animals. Thus, they are preferred by groups which want to avoid and limit the use of animals for ethical reasons.
- For waste reduction: in silico methods do not use chemicals, solvents, etc. (only computers are used). Thus, they are much more environmental friendly.
- Immediacy: in silico methods have the advantage that they only require a computer and, in some cases, Internet access.
- Maximum dissemination: for the above-mentioned easiness, in silico methods have the potential of a huge dissemination, never experimented for any other method. To invest in this direction offers unique opportunities to spread the results of EC funded projects, and attract policy makers and public towards these projects.


The ORCHESTRA project rationale

Several initiatives exist and projects have been funded, to develop new methods and models in line with the so-called 3Rs principle : Replacement, Refinement and Reduction of animal tests.
In this context, the ORCHESTRA project was timely and even urgent. The aim was to disseminate insight from previous EC-funded research on in silico (computer-based) methods for the assessment of physicochemical, (eco)toxicological and environmental properties of chemicals.
In silico methods have a unique role in enabling the evaluation of toxicity from existing experimental data. Specifically, they involve analysing the existing data for a series of related chemicals in order to predict or interpolate the toxicity of other related chemicals. The term ‘in silico methods’ usefully contrasts with in vivo methods, which include experimental tests on live animals, and in vitro methods, which include experimental tests of chemicals on tissue cells in the laboratory. In silico methods can integrate results from both in vivo and in vitro tests. Both in vitro and in silico methods can reduce or replace animal tests, especially when used in combination.
In silico methods involve a manual process called ‘read across’. It involves grouping related chemicals, analysing patterns in the experimental data of their toxicity, relating this to their molecular structure and using this analysis to predict the toxicity of other chemicals. In this process computer programs may be used. Read across uses the underlying concept of Structure-Activity Relationship (SAR): that there is a potentially observable relationship between molecular structure and physical/chemical properties on the one hand, and specific toxicity effects (endpoints) on the other.
QSAR models take this further, by utilising the capacity of computers to analyse complex correlations between molecular structure and observed toxicity for a series of chemicals, they model this relationship mathematically. By being ‘Quantitative’, they are able to predict levels of toxicity for risk assessment or regulatory classification and also to quantify the levels of reliability or uncertainty within a prediction. As each QSAR model is based on toxicity data for a series of related chemicals, it is recommended for use only for related chemicals and only for a particular toxicity endpoint. This restricted domain of use is its ‘applicability domain’.
Leading QSAR models provide the user not only with a prediction for their target chemical, but also with a range of supporting evidence. This enables the user to review the prediction and also to assess its reliability manually, by observing the accuracy of predictions for related chemicals which also have experimental results. In this way the leading QSAR models combine the strengths of computer analysis with manual read across. The new QSAR model platform VEGA, which was a focus of the ORCHESTRA dissemination, is additionally oriented to providing the level of documentation demanded by REACH.
QSAR models and other in silico methods make it possible to evaluate (or ‘screen’) thousands of chemicals in a day. Moreover, the leading platforms developed by EU research and US regulators are freely available online. So they are potentially highly attractive, and can help to make the regulatory demands for chemical safety both achievable and affordable for European industry. In addition (like in vitro methods) they can help to achieve the political, ethical and stated regulatory goals of reducing or replacing animal tests.


The current situation and the ORCHESTRA objectives

Internationally, the use of QSAR models is well-established in the US and Canada. There, the leading QSAR model platforms have been created by the regulatory bodies. Moreover, the OECD has established guidelines and criteria for ensuring quality; these are used in ECHA’s guidance.
However in Europe, evaluation is by industry for acceptance by regulators, so it depends on regulator confidence that industry use of QSAR models can produce reliability, and on industry confidence that their use will be accepted by regulators at EU and national levels. Given the immediacy of the REACH deadlines, a cautious approach by regulators is a focus of considerable frustration in industry. Yet even within industry, a move towards alternative methods involves shifting established institutional practices (and established interests of advisors based in laboratories) which focus on animal testing. The need to generate both confidence and shared understanding has guided this project:
Objective 1: to offer reliable and accessible information that will increase understanding of QSAR models and their uses across the key institutional actors and stakeholders.
Objective 2: to increase stakeholder dialogue and awareness of the concerns and priorities of the other key stakeholders; to enable progress by identifying shared concerns and priorities.
It is not possible for ECHA to approve individual QSAR models, despite clear quality criteria from the OECD. This is because the quality of predictions depends on how they are used in practice. Regulatory caution is therefore partly from a need to verify their use in practice; acceptance is strictly on a ‘case by case’ basis. There is also some regulatory frustration that some QSAR models do not provide the documentation needed. So rather than simply promoting the technology, the priority was to demonstrate reliability in practice, and to ensure they are used wisely.
Objective 3: to inform and enable the wise and rigorous use of QSAR models in practice, by offering users a thorough understanding of their limitations as well as their value.
Objective 4: to identify VEGA as providing the supporting evidence that is needed to review reliability, and so raise expectations of what QSAR models can provide.
Objective 5: to identify VEGA as providing the documentation required by REACH, and so raise expectations of what QSAR models can provide for regulatory use.
Objective 6: to contribute to current understanding of the practical use of QSAR models, by demonstrating and documenting their intended use, and by engaging new and experienced practitioners in using QSAR platforms and reviewing the experience.
The current caution by EU regulators and industry derives not only from the wider challenges of sharing existing data and of producing a ‘weight of evidence’ from varied methods rather than a single animal test, but also from some specific fears about ‘prediction’ and about potentially replacing the expertise of toxicologists with ‘press-a-button’ software.
Objective 7: to address such concerns by emphasising that experimental testing is similarly a ‘prediction’ of human or environmental toxicity, that QSAR models are based on numerous experimental results, and that the complex evidence provided by QSAR platforms means that they must be a tool for the toxicologist rather than a replacement of their expertise.
In industry those who make decisions about what methods to use, or what labs to commission, include industry managers who may not be toxicologists. Similarly the national policy makers are not all toxicologists. These are important audiences in addition to those who are toxicologists.
Objective 8: in addition to (i) generating outputs for toxicologists and (ii) making all outputs rigorous and credible to experts, to focus on (iii) making outputs accessible to decision-makers and other stakeholders who are not toxicologists; (all outputs are also accessible to those without a background in computer modelling.)
These objectives are a significant development of those originally stated in 2009, based on the research and discussions during the project. The more detailed objectives were oriented to individual outputs, and so are best described in terms of those outputs.
ORCHESTRA analysed the outcomes and dissemination strategies of nine EC funded projects chosen as case studies:
- DEMETRA (Development of Environmental Modules for Evaluation of Toxicity of pesticide Residues in Agriculture) – main aim: to develop (Q)SAR models for ecotoxicological assessment – project duration: Jan/2003 to Jun/2006 (42 months).
- CAESAR (Computer Assisted Evaluation of industrial chemical substances according to regulations) – main aim: to develop (Q)SAR models for REACH – project duration: Apr/2006 to Mar/2009 (36 months).
- OSIRIS (Optimized Strategies for Risk Assessment of Industrial Chemicals through Integration of Non-Test and Test Information) – main aim: to develop integrated testing strategies for REACH – project duration: Apr/07 to Sept/2011 (54 months).
- RAINBOW (Research on Animal and In vitro studies and Numerical methods: Bridging Opportunities through a Workshop) – main aim: to organize a workshop about the possibility to integrate in vivo, in vitro and in silico methods – project duration: Mar/2006 to Feb/2007 (12 months).
- HAIR (HArmonised environmental Indicators for pesticide Risk) – main aim: to address the basic question of the organisation of different approaches to assess the impact of pesticides – project duration: Jan/2004 to Mar/2007 (39 months).
- OpenMolGRID (Open Computing GRID for Molecular Science and Engineering) – main aim: to develop a unified platform working in the grid environment for automatic in silico modelling – project duration: Sept/2002 to Feb/2005 (30 months).
- CHEMOMENTUM (Grid services based environment to enable innovative research) – main aim: to extend the experience of OpenMolGrid and shared its main aim – project duration: Jul/2006 to Dec/2008 (30 months).
- CASCADE (Chemicals as contaminants in the food chain: a network of excellence for research, risk assessment and education) – main aim: the target was on food and specific contaminants active on the endocrine system; the aim was not specifically on software, but within its multiple activities, this project aimed at developing screening models for endocrine disruptors – project duration: Feb/2004 to Jan/2009 (60 months).
- SCARLET (Structure-activity relationships leading experts in mutagenicity and carcinogenicity) – main aim: to organise a workshop specific for the use of alternative methods for mutagenicity and genotoxicity – project duration: Jun/2007 to May/2008 (12 months).
Project Results:
The main actions and outcomes of the project have been (i) direct engagement and discussion with a diverse range of potential users of in silico methods and other stakeholders, (ii) direct presentations, education and training during the project, and (iii) the compilation of a range of information that is now available to enable and inform the use of in silico methods within the regulatory framework of REACH. This information is not limited to a theoretical application, but includes, and is informed by, the experience and perspectives of stakeholders from industries and regulatory agencies.
It was clear from initial discussions, that a technical understanding of in silico methods is not the main barrier to their use, so technical information on the methods is not the sole requirement. Instead, the project had to address the wider issues of their regulatory use, and the priorities and concerns of potential users in that context. Similarly, regulatory acceptance is about the use of in silico methods case-by-case, rather that of the technology per se. So almost all the outputs therefore address the issues around use in practice.
The outputs offer information at different levels of complexity, intended to be engaging, accessible and useable for different audiences and functions. Many are written and designed to be accessible to managers, decision makers and stakeholders who are not toxicologists, and whose interest is not to use the technology themselves, but who nevertheless influence the use of in silico methods. (All outputs are intended to be accessible to those without a background in computer modelling.) Most outputs can therefore be understood by the non-toxicologist alongside this report if preferred.
Summary of information and resources available
We list below the different kinds of information that are now available:
Introductory information
1. Introductory leaflet: ‘In silico methods for testing the toxicity of chemicals: an introduction’ (2009) 8-pages, printed in English and French and available online in English, French, Italian and German.
2. The ‘first level’ pages of the ORCHESTRA website, including:
• ‘FAQs’ (www.orchestra-qsar.eu/resources/documents/faq);
• ‘7 Reasons to use in silico methods’ (www.orchestra-qsar.eu/usage-of-in-silico);
• ‘About in silico methods’ (www.orchestra-qsar.eu/about-in-silico-methods) including: ‘Introduction’, ‘Applicability’, ‘Other methods’, ‘The Debate’, ‘Challenges’, ‘Regulatory Context’;
• ‘Glossary’ (www.orchestra-qsar.eu/resources/documents/glossary).
Non-technical explanations of the issues and debates
3. Video documentary: ‘QSARs in REACH? uses issues and priorities. A 30-minute professional briefing based on 20 interviews with regulators, QSAR experts and industry users. (2011) www.orchestra-qsar.eu/resources/videos/qsars-in-reach-video and at
www.qsars-in-reach.researchdissemination.eu;
4. Open access article: ‘The acceptance of in silico models for REACH: Requirements, barriers, and perspectives’. Benfenati, Emilio, Diaza, Rodolfo Gonella, Cassano, Antonio, Pardoe, Simon, Gini, Giuseppina, Mays, Claire, Knauf, Ralf, and Benighaus, Ludger. (2011). Chemistry Central journal: 58. doi:10.1186/1752-153X-5-58.
http://journal.chemistrycentral.com/content/5/1/58;
5. Open access article: ‘Use and perceived benefits and barriers of in silico models for regulatory purposes: Findings from a questionnaire to stakeholders’. Mays, Claire, Benfenati, Emilio and Pardoe, Simon. (2012) Chemistry Central Journal.
www.orchestra-qsar.eu/publications;
6. ‘Summary Report by the Workshop: REACH and QSAR: What can we learn from case studies’. www.orchestra-qsar.eu/documents/279.
Explanations of the technology and its capability
7. ‘Online course: QSAR methods for REACH’.
www.orchestra-qsar.eu/presentations/online-course-qsar-methods-for-reach/intro;
8. ‘E-book: Theory, guidance and applications on QSAR and REACH’. Benfenati, Emilio (2012). www.orchestra-qsar.eu;
9. Open access article reporting a review of the use of QSAR platforms by expert users: ‘Using toxicological evidence from QSAR models in practice’. Benfenati, Emilio, Pardoe, Simon, Martin, Todd, Gonella Diaza, Rodolfo, Lombardo, Anna, Manganaro, Alberto and Gissi, Andrea. (Forthcoming). www.orchestra-qsar.eu/publications;
10. Conference presentations at SETAC Europe 2011, presented on video
www.orchestra-qsar.eu/qsar-presentations-setac-europe-2011-full-videos;
11. A range of further articles intended for toxicologists and QSAR users, some of which are open access. www.orchestra-qsar.eu/publications.
Invitations to collaborate in further development and review
12. Listing of the VEGA platform as a solution to reducing and replacing animal testing, and invitation for review and further development across institutions. www.crackit.org.uk/share/crackitsolutions
Resources for users and potential users of QSAR models
13. Direct links to the VEGA, CAESAR and DEMETRA platforms
www.orchestra-qsar.eu/resources/information

The ORCHESTRA Workshop on in silico methods
The organization of a dissemination workshop about in silico methods was one of the key objectives of the ORCHESTRA project.
This event was aimed at involving stakeholders (mainly from industries and regulatory bodies) within ORCHESTRA.
Therefore, at the dissemination workshop “REACH and QSAR – What can we learn from case studies?” held in April 6th 2011, in Milan, Italy, experts and stakeholders were invited to hear more about the latest research on in silico methods, as an alternative to animal testing for determining substance hazards and their possible use within the European regulation REACH.
112 experts and stakeholders from various groups and countries participated. They were addressed from:
• Regulators from all EU countries, from national, European and international regulatory agencies,
• Chemical industry (small, medium and large enterprises),
• Laboratories and consultants involved in the REACH process.
The objectives
The overall objective of the workshop was to initiate a dialogue between the project members themselves and the EU-Stakeholders. The workshop highlighted the different concepts and methods developed by EC research projects as well as their main results and outcomes, focusing on the benefits for stakeholders. The main topic was the match between REACH requirements and methods developed by the projects considered within ORCHESTRA.
More in details, the main goals of the workshop were:
• To listen to updated news from ECHA about how QSAR models have been used for the dossier preparation and first expe¬rience from the registrations of substanc¬es;
• To explore the use of QSAR for registra¬tion, classification (CSA) and prioritization;
• To discuss QSAR models with industry and regulators from all member states.
Moreover, during the workshop the members of ORCHESTRA project gave talk, presentation and had face-to-face discussed with participant, aiming at:
• Present tools developed within EC projects which proved to give good per¬formance;
• Show practical examples of in silico/QSAR models applied for REACH;
• Discuss the uncertainty and limitations of in silico methods.
• Query and facilitate discussion be¬tween regulators and industry;
• Collect feedbacks and suggestions from practice and ideas for the improvements of in silico methods in the REACH context.
The workshop agenda
The workshop was organized in three main sessions:
Opening and welcome: ORCHESTRA and the workshop were presented by members of the project. A representative of the Italian ministry of environment also gave a brief welcome presentation.
• “ORCHESTRA in a nutshell – what can we learn?”, Simon Pardoe (PublicSpace Ltd, Lancaster, United Kingdom). Main topics were: (1) “Are QSAR models new and untried?” (2) “Do you just press a button and get an answer?” and (3) “Do QSAR models produce results that are less reliable than an animal test?”.
Use of In Silico methods: This session was aimed at giving the participant an introduction about in silico methods and their potential application within the REACH regulatory framework. Moreover, participants had the opportunity to learn how in silico models had been used in dossier preparation for REACH, directly from an ECHA representative. Two presentations were given during the first session:
• “Use of in silico Methods is multifunctional: Some examples of tools developed within EC project”, Emilio Benfenati (Istituto di Ricerche Farmacologiche Mario Negri, Milan, Italy). The main aim was to emphasize the seven reasons to use QSAR models:
1. Innovation (also in view of thousands of new data - ToxCast);
2. Time for experiments;
3. Occurrence of enough laboratories/ resources;
4. Reduction of costs;
5. Use of animals;
6. Prioritization needs;
7. Pro-active approach for safer and ‘greener’ chemicals;
• “How QSAR has been used for the dossier preparation? Lessons from the registrations of substances so far done“, Doris Hirmann (European Chemicals Agency - ECHA, Helsinki, Finland).
Ludger Benighaus (ZIRN/University of Stuttgart, Germany) acted as facilitator during a round table, asking specific questions to five selected representatives of regulators and industries.
Tools and Instruments of In Silico Methods: The afternoon session were organized in working groups, each one focusing on a case study. After an introductive talk given by an ORCHESTRA member, participants had to discuss the possible applications of QSAR models.
Workshop Outcome
Representatives from industry, regulators and consultants discussed how in silico methods could be transferred into prac¬tice. The most important point was to improve the communication and the understanding of the models to facilitate all the actors involved in the registration of the substances for REACH.
The experts explained that non-test information is a valuable tool to use in the REACH context and some of the models can give good predictive performance. They suggested using integrated testing strategies, which are supported by REACH. A combination of (Q)SAR with read-across, in vitro tests and in vivo data in a weight of evidence approach is valuable, but achieving its broad use will require a mind shift in the community.
They also advised that in silico methods should be used to predict safety profiles of discrete chemicals for priority setting at the early stage of product develop¬ment, for mechanistic understanding, to prepare a dossier submitted to regula¬tory authorities or for classification and labelling. However, the alternative tests are diffi¬cult to apply to complex endpoints such as reproductive toxicity.
Both in¬dustry and authorities asked for more case studies and more successful examples of good use of QSAR models for the registration of chemicals in REACH. Regarding the acceptance follow¬ing very strictly the REACH text and the guidance documents were highlighted to be essential. But if the cases show good results the use and acceptance will be higher in future.
Suggestions for Cooperation within REACH and QSAR
Most participants agreed that there should be closer cooperation between REACH and QSAR. As for how this cooperation could be organ¬ized, in particular, the following suggestions were raised:
• Knowledge-building by further training/courses/workshops;
• More information/guidance;
• Deeper/better involvement of authorities;
• Regular meetings between participants and the authorities;
• More intra-industry cooperation;
• Better coordination of different initiatives.
Central Recommendations from Experts and Stakeholders
• QSAR as a key study or supporting information: Non-test in¬formation is a valuable tool to use in the REACH context and can be predictive. Furthermore, in silico methods or QSAR can be seen as a support offering information on chemicals, delivering supporting evidence or as a key study as well in the REACH evaluation.
• QSAR used for C&L, registration and prioritization: The use of QSAR is conditional and the models should be used only if the scien¬tific validity of the model can be demon¬strated and applicability of the model is analysed. In silico methods were suggested for use in the registration phase, as well in the classification & labelling or prioritization phase of the REACH process. In silico methods should be used to predict safety profiles of discrete chemicals for priority setting at the early stage of product develop¬ment.
• Case by case basis and more: ECHA policy is that acceptance of in silico methods should be done on a case by case basis in the current stage of development. It is necessary to use in silico methods to gain more experience and get more scientific validity. ECHA is requested to evaluate the substances, and not the QSAR models. The detailed evaluation of the case studies, as for in¬stance started within the working groups on BCF and mutagenicity, can provide material useful to better define limits of applicability.
• A common language: Not only more experience is essential, but it is also fundamental to establish a unique and comprehensive language between the actors. There are still communication problems be¬tween developers, industries and toxi¬cologists which use the same words but mean different contents.
• More training in models: Another point was the different interest of various actors. Regulators want to lead the attention to toxicity endpoints. In or¬der to achieve more acceptances, more training and transparency in the applica¬bility and reliability of models would be helpful.
• Closer cooperation with ECHA: The guidance documents are essential in the evaluation process of the meth¬ods. A closer cooperation in the evalu¬ation process of ECHA with industry is necessary to ensure that industry and regulation think in the same manner.
The full Workshop Summary Report is available on line and can be downloaded from the ORCHESTRA website.
The ORCHESTRA web portal
The development of the official web portal was another key objective of the ORCHESTRA project. The main purpose was to build an on line reference resource for in silico methods, with contents specific for different type users. The ORCHESTRA website has been the main way for communicating results and updates as well as for the promotion of in silico resources and events. Furthermore, the web portal has also been an important way for gaining information from stakeholders, thanks to the on line surveys and exercises.
The ORCHESTRA web portal has been made available through three addresses:
• http://www.orchestra-qsar.eu/
• http://www.in-silico-methods.eu/
• http://www.insilicomethods.eu
Whereas the first one regards the project itself, the other two were chosen to reflect the aim of the website to be a reference hub for in silico methods.
The final and current version of the web portal has been organized as follow.
The homepage
The homepage of a website is commonly designed to be a sort of showcase, highlighting “what’s new” and “what is important”. The ORCHESTRA homepage has been designed in this way. Key features are:
• Presence of animated sections, which scroll between news and updates;
• A tabbed section about documentaries, learning materials, highlights and surveys;
• “Dissemination information and resource”, which provide introductory and training materials, as well as a site map of the most important sections of the ORCHESTRA web portal.
About ORCHESTRA
This section provides an overview of the project, explaining which are the main aims and whom is addressed by its results (the stakeholders). A section about the members of the project is also obviously present as well as the contacts of the project.
About in-silico
In silico methods are introduced and explained in this section, which also highlights other sections and resources valuable for understanding these methods (e.g. the introductory leaflet, FAQs, documentaries, etc.).
Within its subsections, this area of the web portal is aimed to provide information about:
• Definition of in silico methods, mainly focussing on (Q)SARs;
• The areas of applicability of these methods;
• Comparison on in silico with other approaches;
• The debate regarding the use of these methods for regulatory purposes;
• What in silico methods offer;
• The regulatory contexts.
Why use in-silico?
Seven reasons for using in silico methods have been identified and are here depicted:
1. Cost and feasibility
2. Innovation
3. The focus on ‘weight of evidence’
4. Animal testing as ‘a last resort’
5. Prioritisation
6. Pro-active decision-making
7. The future of toxicology
Resources
This is probably the most important and interesting section of the ORCHESTRA web portal and contains the results of the project’s effort in the dissemination of in silico methods.
The main page is organized as a sort of secondary homepage; the showcase-structure has been also here adopted, highlighting what’s new in its subsections, which are:
• The collection of all the video documentaries produced by the project, containing interviews with stakeholders (e.g. regulators such as ECHA), footage from international events, introductions on in silico, results from the project, etc.;
• The informative documents produced within ORCHESTRA, as well as reports of important outcomes and of the collaboration with stakeholders;
• The list of publications regarding in silico methods, written within ORCHESTRA project or by project members with acknowledgment to ORCHESTRA;
• The learning material produced, organized in subsections:
o Introductory leaflet, written in four languages (English, French, German, Italian);
o Online course;
o Video documentary “QSARs in REACH? - Uses, issues and priorities”;
o Frequently Asked Questions (FAQ);
o Glossary;
• The software section, with link to free QSAR models potentially usable for regulatory purposes;
• A list of databases containing experimentally determined properties of chemicals;
• A list of other useful links.
News & Activities
As suggested by its title, this section shows all the most recent updates and news, such as new material published in the web portal.
A calendar page is also present, showing important dates and events.
Get involved
This is another important section of the ORCHESTRA web portal. It has been designed to be stakeholder-type oriented; users can also participate and interact with the project.
• The communities subsection is organized to specifically show the content of the ORCHESTRA to the four main communities of user identified: industry, regulators, academia, NGOs.
• One of the main way for users to interact and contribute, were through the compilation of on line questionnaires. The survey subsection contains these questionnaires as well as the results obtained.
• All the external activities have been organized in a specific subsection, in turn organized in: Courses & Seminars and Workshops & Annual meetings.
Video Documentary: ‘QSARs in REACH? - Uses, issues and priorities’
The intention of the video documentary was to provide an authoritative and engaging account of the issues and priorities in using QSAR models within REACH. Those issues and priorities are voiced directly by key experts and stakeholders.
The documentary offers a 30-min professional briefing, based on 20 interviews with key regulators, industry representatives and expert QSAR developers. It addresses current limitations, issues and priorities. It is intended primarily for toxicologists, regulators and developers of QSAR models, though it is also accessible to a wider audience.
Twenty five interviews were conducted and twenty were used in the video. The four core expert interviews (with Professor Wim de Coen at ECHA, Professor John Dearden, Dr Nick Price and Dr Emilio Benfenati) were prepared from detailed discussions of issues with Dr Emilio Benfenati and from detailed reading of the relevant literature – including information and published academic articles written by the interviewees. Interviews were conducted in two companies in Milan, with extensive location filming to provide images which help to show what the various interviewees are talking about. Further interviews with industry representatives and regulators were conducted during the ORCHESTRA Workshop. In each interview, the interest was in people’s insight, experience and current thinking.
The interviews were ‘semi-structured’, to create dialogue that could offer the kind of engaging detail and clarity needed for the documentary. For the on-site interviews, questions were discussed with interviewees beforehand, and developed during the interviews. Responses were frequently filmed a second time when an issue was clarified for the interviewer and/or the interviewee found a more concise or succinct way of responding.
Specifically, the interviews were an opportunity to put to each expert/stakeholder the concerns and comments made by earlier interviewees, especially interviewees from the other sectors. This was essential so that the video not only raises concerns but also offers responses to those concerns. Like the intended viewer, the ORCHESTRA partner making the film wanted to hear and include both the concerns and the informed responses to them. This active engagement is reflected in the interviews, making them more interesting and engaging for the audience. These are not rehearsed statements, but responses to sometimes challenging questions.
The subsequent process of constructing, developing and editing the video was based on an analysis and distillation of the comments, rather than a pre-planned script. That is central to making the video an independent, interesting and evidence-based documentary rather than the promotion of a particular viewpoint. It was reviewed by all the main contributors, and formally approved by the ECHA Communications Unit.
The documentary includes introductory explanations and questions from a presenter. It can be viewed in four separate sections, each offering a different perspective. (These are separate videos in the online version.)
• Introduction (1 min) & Regulatory perspectives (8 mins): Interviews with regulators, including ECHA and national regulators on how they see the role and potential of QSAR models within REACH.
• What makes a good QSAR model? (9 mins): Interviews with QSAR developers on the value and limits of QSAR models, offering an understanding of the core issues of what makes a good QSAR model, the importance of using models appropriately, and how to review or question the outputs.
• Industry perspectives (8 mins): Interviews with industry representatives on the ways in which they are currently using QSAR models, and their current concerns and priorities for enabling wider use within REACH.
• Future directions & List of contributors (4 mins): Comments from all stakeholders on future directions.
The documentary has been used within the project both as a key form of dissemination and as a support to other tasks in other work packages:
1. A video installation at the 2011 SETAC Europe conference: the documentary was first designed as an installation for the SETAC 2011 Europe conference as part of the ORCHESTRA SETAC booth; it is therefore explicitly designed for viewing non-sequentially at a public event.
2. Online streaming on the ORCHESTRA website and on Vimeo: the documentary is streamed on the ORCHESTRA website homepage, and on a dedicated page for the documentary. It significantly increases the engagement value of the ORCHESTRA website, by bringing the issues and institutions to life through the images and interviews. It is also streamed on a dedicated site with a concise summary introduction to explain the documentary.
3. Availability and distribution on DVD: A DVD version is designed for use as an installation at conferences and other professional events. It can be watched selectively in sections, for use in training, meetings or presentations. It comes with a poster so that organisations can publicise access to it online and on disk. For example, after the SETAC Europe 2011 conference the DVD was requested and used by the European Chemical Industry Council (CEFIC) in 2011 for discussion prior to a meeting with ECHA. It has also been used by a QSAR specialist as part of a workshop at ECHA.
4. A resource for platform presentations: the section ‘What makes a good QSAR model?’ was included within two PowerPoint presentations to a UK toxicology conference and to an animal-protection NGO.
View at: http://www.in-silico-methods.eu/resources/videos/qsars-in-reach-video and at
www.qsars-in-reach.researchdissemination.eu.
Video presentations, interview and report
Seven further videos are streamed on the ORCHESTRA website. Of these, five videos make conference presentations at the SETAC Europe 2011 conference available to a wider audience. They are:
• “How to improve the safe use, explanation and acceptance of QSAR models for REACH?” – Emilio Benfenati (18 mins);
• “New QSAR models for regulatory purposes” – Giuseppina Gini (27 mins);
• “Survey on Stakeholder Perceptions of Benefits and Barriers to the Use of in silico Methods: Reflections on Regulatory Aspects” – Claire Mays, Emilio Benfenati, Simon Pardoe and Ralf Knauf (13 mins);
• “How can the cautious policy of acceptance 'case-by-case' become a mechanism of progress towards the rigorous, successful and extensive use of QSARs within REACH?” – Simon Pardoe, Emilio Benfenati, Giuseppina Gini, Claire Mays (25 mins).
• “The development of new tools towards a better exploitation of non-testing methods for regulatory purposes”, from ANTARES project – Alessandra Roncaglioni, Alberto Manganaro, Rodolfo Gonella Diaza, Anna Lombardo, Maria Petoumenou and Emilio Benfenati (27 mins).
View at: www.orchestra-qsar.eu/qsar-presentations-setac-europe-2011-full-videos.
A further short video simply offers a visual report of engagement with toxicologists in the ORCHESTRA booth at the same conference.
View at: www.orchestra-qsar.eu/news/qsar-booth-setac-europe-2011-brief-visual-report
A further video is a 7-minute version of an interview with Professor Wim de Coen, Head of Evaluation 1 at the European Chemicals Agency (ECHA), conducted on 30 June 2010. It was originally created to be shown to delegates at the opening of the ORCHESTRA workshop.
View at: www.orchestra-qsar.eu/resources/videos/interview-wim-de-coen
The ORCHESTRA e-book
One of the final and most ambitious task of the project was the development of the e-book “theory, guidance and applications on QSAR and REACH”.
The e-book is composed by three main sections and seven annexes:
• Part A. Theory
Benfenati E., Gonella Diaza R., Gini G., Cardamone L., Gocieva M., Mancusi M., Padovani R. and Tamellini L.
• Part B. Practical cases
Benfenati E., Gonella Diaza R. and Gissi A.
• Part C. The applications and the possible uses of QSAR models
Benfenati E., Gonella Diaza R., Gini G., Cardamone L., Gocieva M., Mancusi M., Padovani R. and Tamellini L.
• Annex 1. QSAR and SAR models: Basic definitions
• Annex 2. An introduction to toxicology
Gini G., Cardamone L., Gocieva M., Mancusi M., Padovani R. and Tamellini L.
• Annex 3. Rigid and flexible topological indices: variety of the representations of the molecular structure
• Annex 4. A free and open source informatics library for chemistry: Chemistry Development Kit (CDK)
• Annex 5. REACH
Gini G., Cardamone L., Gocieva M., Mancusi M., Padovani R. and Tamellini L.
• Annex 6. Genetic Algorithms in QSAR for REACH
• Annex 7. The CORAL software: principles, results, perspectives
The ORCHESTRA e-book has been organized in three main parts in order to introduce in silico methods for beginners and people with low or no experience in this field.
Part A – Theory
The book starts with a theoretical introduction to in silico methods, focusing in particular on (Q)SARs. It gives an overview on how the models are built, from the choice of the dataset used to build the model and some examples of common modelling algorithms, to the processes of validation of these models. This first part also introduces some advanced modelling approaches such as hybrid models. A brief comparison of in silico approaches versus other approaches is also given.
Part B – Practical cases
In this section, the five model developed by the CAESAR project are taken as examples, explaining deeply how they have been built, including the decision taken about the dataset to use, how it has been split between training set (molecules used to train the model) and test set (molecules used for the model validation). The performances of the models on training and test set are also reported.
Apart from the model itself, the endpoint and its relevance within the REACH legislation are reported as well as the application of (Q)SAR models for this particular endpoint within the REACH framework.
Part C – The applications and the possible uses of QSAR models
This section of the e-book explores the possible uses of QSAR for regulatory purposes and has been designed especially for people working in the chemical industry. It gives an overview of the use of in silico methods in REACH, focussing on the requirements under this legislation.
This e-book can be seen as a sort of first step, the initial way to distribute and promote the information produced or gained during the project’s lifetime. Istituto Mario Negri (the leading partner) has planned to maintain this e-book after the end of ORCHESTRA, updating it with more and more material. Moreover, the main aim for the future will be to convert this e-book in a sort of mini and dynamic website which will be easier to extend with more information.
The technical survey for experts: benefits and barriers to the use of QSAR methods
During the period September 2010 – April 2011, experts and stakeholders from industry, consultants, regulatory bodies and academia were invited to answer to an on line “technical” questionnaire on “benefits and barriers to the use of QSAR methods”. The response rate was about 10%, yielding 33 replies.
In order to gain some information from those who didn’t respond to the questionnaire, the ORCHESTRA consortium decided to send a far more shorter one, simply asking if they have ever used in silico methods or not, and if not, the reason. This second short questionnaire resulted in another 29 answers.
Most of the respondents stated that they have used in silico methods and cited a total of 28 tools. The three most cited tools were: OECD Toolbox, U.S. EPA EPIsuite and CAESAR. Moreover, human toxicity endpoints resulted to be among the more assessed using in silico methods.
• Academia and consultants (ACACON) utilize QSAR more for physico-chemical property evaluation and seem to move in the direction traced by regulators, planning an increased effort in using these methods in human toxicity endpoints. The most important benefit of in silico tools is the time-saving opportunity which characterizes these methods. Moreover, ACACON seems to be relatively confident in using QSARs for regulative purposes (e.g. risk assessment and CLP). ACACON have observed that results of in silico predictions may be sometimes not enough for toxicologists, for example in determining properties of mixtures and assessing long term toxicity. Moreover, they ask for more additional information which may help in increasing the confidence of QSAR predictions.
• Chemical manufacturers or their organization (INDUS) appear to “lag” behind the other categories in the application of in silico methods. They are interested in these methods both for their time-saving characteristics and as tools for providing supporting information. INDUS seems to agree with ACACON in asking for more additional information, which may increase the reliability of the predictions. Moreover, INDUS are not interested in better understanding how the models work, they ask REGUL to provide clear information about which models can be safely used and in which cases.
• Regulatory bodies (REGUL) seems more interested in human toxicity endpoints and are planning to increase their attention in environmental fate and ecotoxicology endpoints. In silico approaches, though, are viewed more as supporting tools. Unlike the other category, computational method haven’t been used as prioritizing tools, however there is an agreement that they will gain more and more importance in the future.
The complete questionnaire can be examined through the ORCHESTRA website at the following address: http://www.orchestra-qsar.eu/survey-summary-benefits-and-barriers-to-qsar.
ORCHESTRA booth at SETAC 2011
Among the dissemination strategies chosen since the very beginning of the project, it was planned that ORCHESTRA members were going to participate to important international events, giving talks, lessons and/or presenting posters.
The SETAC (Society of Environmental Toxicology and Chemistry) annual meeting is undoubtedly one of the most important annual events. ORCHESTRA members participated to three annual meetings: Seville 2010, Milan 2011 and Berlin 2012 (more information in the “International events participation” section of this report).
The SETAC Europe 21st Annual Meeting was organized in Milan, by Emilio Benfenati (Istituto Mario Negri) and took place on May 15th – 19th 2011. There have been more than 2300 participants. ORCHESTRA took advantage of this event to book a booth with the aims of presenting the project and its results as well as to show some examples of in silico models and let interested users to try them, with expert support from members of the ORCHESTRA project:
Six posters were presented at the booth:
• Sources and data quality of the existing data for Bioconcentration (OSIRIS project);
• Applicability and performance evaluation of QSAR models for Bioconcentration in fish (OSIRIS project);
• Integrated Testing Strategies (ITS) to optimise the assessment of bioconcentration under REACH framework (OSIRIS project);
• An uncertainty-based framework for the integration of data from different sources in ecotox assessment (OSIRIS project);
• Stakeholder Perceptions of Opportunities and Challenges for the Use of In silico Methods in Europe (ORCHESTRA project);
• A Scheme for the Analysis of the Results and Dissemination & Exploitation Activities of In silico Models (ORCHESTRA project).
A TV screen equipped with several headphones was also present at the booth, showing the video documentary: “QSARs in REACH?” (more info in the “Video documentary” section of this report).
Finally, to present in silico models, two PCs were installed at the booth. Members of ORCHESTRA were present to briefly introduce in silico models in toxicology to interested people and to guide them in the direct use of CAESAR, VEGA and DEMETRA software.
Meet the German regulators: seminars and exercises
Several seminars were organized for regulatory agencies, presenting ORCHESTRA and the possible use of QSAR models in toxicology.
The meeting held at the German’s Federal Environment Agency (Umweltbundesamt – UBA) in Dessau-Roßlau, Germany on November 22nd 2011, was of particular interest and importance.
Objectives and Agenda
The workshop was focused on regulatory aspects, so the participants were agency staff of the UBA itself, mainly of the 4th Division - Chemical and Biological Safety.
The main objectives of the workshop were:
• To present the ORCHESTRA project and different QSAR-models and the VEGA-platform in regards of regulatory aspects;
• To understand and discuss with regulators how one can use QSAR for environmental endpoints and where the limitations of this approach are, especially how the documented data should be interpreted.
Part 1 of the workshop: Presentations
In the first part of the workshop Ralf Knauf gave an overview about the project ORCHESTRA itself, while Emilio Benfenati and Giuseppina Gini introduced QSAR-models in regard to regulatory aspects and the VEGA platform, which provides online access to several QSAR models for regulatory purposes.
Part 2 of the workshop: Exercises on case studies
The second part was the most interesting one since it was focused on case studies and the discussion about the possible acceptance of the documented results. The participants were given the possibility to have a closer look on how VEGA presents the results and give feedback about their usability.
The aim was not to teach the participants the handling of the VEGA platform but to show them how to interpret the provided data, so they were able to decide:
• If the information given by the model was useful or not and
• To give feedback on the software itself, its way of documenting the results and most of all on the acceptance of those results regarding regulatory issues.
To support the demonstration on screen, printed results of six chemicals analysed by VEGA were distributed varying in the clearness of prediction. After a short introduction of the VEGA user interface and the structure of the documentations, the participants, guided by experts from ORCHESTRA project, approached to interpret the results.
Workshop Outcome
The overall feedback tended to be very positive. The participants appreciated especially the way VEGA documents the results and that it allows the user to combine the prediction by a QSAR model with the read-across approach. In their experience, a possibility missed most of the time. The information given by VEGA provides a good balance between automatically created information and required expert/user decision. The outcome of the models forces the user to think about the results and to interpret them.
Remarks
A particular interesting remark was that the acceptance of QSAR results depends on the context and the legislative context. Participants agreed that acceptance increases if QSAR results are used in a weight of evidence approach taking all available information (QSAR as well as read-across and additional information) into account.
Acceptability of QSAR results: an on line exercise
The purpose
Starting from the experience gained during the face-to-face exercise proposed during the seminar given at the German Federal Environment Agency, an on line exercise on the acceptability of results obtained with in silico methods was developed.
The purpose (stated clearly to participants) was to obtain feedback from regulators, industry, consultants, researchers and experts within academia, on the reliability and therefore acceptability of results from QSAR models, by considering practical case studies.
The method
In order to keep the evaluation free from possible opinions on the models themselves, the users were not asked to use them. The predictions were performed by ORCHESTRA members, and the outputs were directly provided within the exercise.
The selected endpoint for this exercise was the bioconcentration factor (BCF), which is one of the most relevant within REACH regulation; three free software with predictive models for BCF were chosen:
• The VEGA platform – http://www.vega-qsar.eu/;
• US EPA’s Estimation Program Interface (EPI) Suite – http://www.epa.gov/oppt/exposure/pubs/episuite.htm;
• US EPA’s Toxicity Estimation Software Tool (TEST) – http://www.epa.gov/ordntrnt/ORD/NRMRL/std/qsar/qsar.html.
Moreover, three molecules were selected as case studies in order to provide users with three example of prediction reliability: good, uncertain and low. The reliability is assessed by the models themselves, based on integrated parameters such as the applicability domain (AD) evaluation and the performances of the model with similar molecules present in the model database and for which the models knew the experimental values.
Therefore, nine reports were provided to the participants, who had to answer to a simple question: “is the value obtained for the model XXX reliable?”. The exercise also asked a brief motivation of each answer.
Participants were also asked their affiliation type:
• Industry/Consultant for industry;
• Academia/Research institute;
• Public agency;
• Other.
They were also asked whether they had previous experience of using QSAR models, as a user, as a reviewer/regulator or as a developer. Finally, they were asked whether or not they had ‘experience of the evaluation of BCF for regulatory purposes’.
Results
A total of 28 replies were received from 11 countries: Austria, Belgium, France, Germany, Italy, Portugal, Slovenia, Spain, Switzerland, UK, and USA.
The introductory leaflet on QSAR
An introductory leaflet on in silico methods and their possible application for regulatory use was developed and made available in printed version and through the ORCHESTRA web portal.
The leaflet is organized into six sections:
• Testing the toxicity of chemicals, highlights the importance of obtaining as much as possible information about the risks of chemicals we commonly use;
• REACH: the challenge, gives a brief overview about REACH, its requirements and the clear statement about the use of alternative methods (including in silico) for the reduction of animal testing;
• What are in silico methods, it’s an introduction to the computer based methods written to be comprehensible for people who probably haven’t even hear about them;
• Advantages of in silico methods, explains which are the main advantages in using these methods in toxicology for regulatory purposes;
• Current limitations in the use of in silico methods, underlines which are the problems related to the use of in silico methods for regulatory purposes;
• Find out more, provides information about websites and resources available for learning more about in silico methods;
• Glossary, explains the meaning of the main and most basic terms related to these methods.
This leaflet has been translated into four languages: English, Italian, German and French.
The on line course on QSAR
During its lifetime, ORCHESTRA produced introductory material (such as the leaflet described above) and gained information about in silico methods, directly from the main actors involved in the regulation of chemicals in Europe (such as regulatory bodies and industries).
Moreover, several events were organized to present and explain the use of in silico methods, directly in the headquarters of stakeholders. ORCHESTRA members also participated to international events, presenting in silico resources and results from the project.
With the idea of creating a public available resource for understanding and/or deepening the knowledge of in silico methods and how they can be used for regulatory purposes, all the knowledge about these methods has been translated in an on line course, named “Course on QSAR methods for REACH”.
The course consists of a series of slides, organized in four main sections:
• Part 1 – Theory about QSAR Methods
• Part 2 – The Use of QSAR Models for Regulatory Purposes
• Part 3 – European Projects and QSAR Models
• Part 4 – The VEGA Platform: A QSAR Application
Participation and organisation of events and courses
ORCHESTRA members participated to several international events, giving talks about in silico methods, presenting posters and/or organizing courses.
International events
2009 September 2nd, Rome, Italy. World Congress on Alternatives & Animal Use in the Life Sciences. Poster: ORCHESTRA: a new EC project to link the research of in silico models with users’ needs. E. Benfenati, G. Gini, I. Malerba, V. Castiglioni. This Poster received an award.
2009 November 6th, Brussels, Belgium. EPAA Conference. Poster: How to improve availability and acceptance of in silico models. The ORCHESTRA EC project. E. Benfenati, R. Gonella Diaza, C. Milan, V. Castiglioni, G. Gini.
2009 November 24th, Istituto Mario Negri, Milan, Italy. Course on “QSAR models” to industry users. E. Benfenati.
2010 January 14th, Istituto Mario Negri, Milan, Italy. Lesson on “QSAR models” within the Master Course on “Master in Risk Assessment and Risk Analysis”. E. Benfenati.
2010 February 9th, Istituto Mario Negri, Milan, Italy. Course on QSAR, organized by Federchimica. E. Benfenati,
2010 February 24th, Milan, Italy. Workshop “The LRI Long Range Research Initiative”. Presentation: New development in predictive computational models for toxicity of chemicals. E. Benfenati.
2010 May 23rd – 27th, Seville, Spain. SETAC Europe 20th Annual Meeting. Poster: Disseminating the QSAR models: the ORCHESTRA EC project. E. Benfenati, R Gonella Diaza, A Cassano, V. Castiglioni, G. Gini, S. Pardoe, M. Poumadère.
2010 May 11th, Istituto Mario Negri. Milan, Italy. Course on QSAR, organized by Federchimica. A. Roncaglioni
2010 May 24th – 28th, Montreal, Canada. 14th International Workshop on QSARs in Environmental and Health Sciences. Poster presented by partner IRFMN.
2010 May 30th, Postdam, Germany. OpenTox Workshop: “Integrating Predictive Toxicology Resources and Applications”. Presentation: Perspective on infrastructure requirements for chemistry research in predictive toxicology. E. Benfenati.
2010 June 16th – 17th, Stresa, Italy. Workshop: “Integrating New Advances in Exposure Science and Toxicity Testing: Next Steps”. Talks with regulators and industry. Participation: E. Benfenati, R. Knauf.
2010 June 29th, Helsinki, Finland. Meeting with ECHA. Presentation: Towards a safer and more transparent use of QSAR models for toxicity prediction. E. Benfenati.
2010 July 23rd, Rome, Italy. Comitato Tecnico di Coordinamento per il REACH. Talks with regulators and presentation of the project. E. Benfenati.
2010 September 23rd, Helisinki, Finland. Meeting at ECHA. Participation: E. Benfenati.
2010 Otober 1st, Lana (Bolzano), Italy. Workshop: “La salubrità ambientale: nuovi scenari scientifici e nuovi adempimenti alla luce della regolamentazione REACH”. Presentation: La regolamentazione REACH e l’uso dei modelli predittivi per la tossicità. E. Benfenati.
2010 October 27th, Parma, Italy. Meeting with EFSA, and presentation of the project ORCHESTRA. E. Benfenati.
2010 November 17th – 18th, Brussels, Belgium. Cefic-LRI 12th Annual Workshop “Reduction of uncertainty Enabling Decision Making”. Presentation: Predictive models and their appropriate use within the applicability domain. E. Benfenati.
2010 November 26th, Milan, Italy. Workshop on: “Toxicity Testing in the 21st Century and Alternative Methods”. Presentation: How reliable are in silico methods for toxicity prediction?. E. Benfenati.
2010 December 13th, Rome, Italy. Workshop on: “Le attività di ricerca ISPRA nell’ambito del regolamento REACH”. Presentation: I metodi di non testing per la stima delle proprietà (eco)tossicologiche. A. Roncaglioni.
2010 December 15th, Istituto Mario Negri, Milan, Italy. Course on QSAR, organized by Federchimica. E. Benfenati and A. Roncaglioni.
2011 February 3rd, Milan, Italy. “6 Conferenza Sicurezza Prodotti: REACH”. Presentation: I primi risutlati dei progetto eruropei ORCHESTRA e ANTARES sui metodi alternativi. E. Benfenati.
2011 February 22nd, Milan, Italy. Workshop: “Risk assessment. Metodi di valutazione per l’ambiente e per l’uomo”. Presentation: Predizione di proprietà tossicologiche e ambientali in caso di mancanza di dati da letteratura su sostanze chimiche. A. Roncaglioni.
2011 April 11th – 15th, Paris, France. IEEE Symposium Series on Computational Intelligence. Submitted paper: Mining Toxicity Structural Alerts from SMILES: A New Way to Derive Structure Activity Relationships. T. Ferrari, G. Gini, N.G. Bakhtyari, E. Benfenati.
2011 May 16th – 19th, Milan, Italy. SETAC Europe 21st Annual Meeting.
Presentation: How can the cautious policy of acceptance ‘case-by-case’ become a mechanism of progress towards the rigorous, successful and extensive use of QSARs within REACH?. S. Pardoe.
Presentation: Integrating statistical and structural results in predictive modelling for a multi-view evaluation. G. Gini.
Presentation: Regulators’ role and mission: Feedback from various stakeholder contexts. C. Mays.
Presentation: How to improve the safe use, explanation and acceptance of QSAR models for REACH?. E. Benfenati.
Presentation: The development of new tools towards a better exploitation of non-testing methods for regulatory purposes. A. Roncaglioni.
Poster: Stakeholder perceptions of opportunities and challenges for the use of in silico methods in Europe. C. Mays, S. Pardoe, E. Benfenati and R. Knauf.
Poster: A scheme for the analysis of the results and dissemination and exploitation activities of in silico models. E. Benfenati, R. Gonella Diaza.
2011 August 28th – 31st, Paris, France. EUROTOX 2011 - 47th Congress of the European Societies of Toxicology. Participation by partner IRFMN.
2011 September 3rd – 7th 2011, Maribor, Slovenia. 6th International Symposium on Computational Methods in Toxicology and Pharmacology Integrating Internet Resources (CMTPI 2011). Presentation by partner IRFMN.
2011 September 13th, London, UK. SETAC UK annual meeting “Polluted planet: sustaining ecosystems and biodiversity”. Presentation by partner PUBLICSPACE.
2011 October 17th – 18th, Leipzig, Germany. Expert Meeting of EUROECOTOX on the use of alternative methods. Participation by partner IRFMN.
2011 October 26th – 27th, Berlin, Germany. 6th Meeting of the OECD QSAR Toolbox Management Group. Presentation by partner IRFMN.
2012 May 20th – 25th, Berlin, Germany. SETAC Europe 22nd Annual Meeting - 6th SETAC World Congress. Presentation by partner IRFMN.
2012 June 18th – 22nd, Tallinn, Estonia. QSAR2012. Presentations and Posters by partners IRFMN and POLIMI.
Didactic courses and initiatives
2010 January 14th, Istituto Mario Negri, Milan, Italy. Lesson on QSAR models, within the Master Course in Risk Assessment and Risk Analysis. E. Benfenati.
2010 May 23rd, Seville, Spain. SETAC Europe 20th Annual Meeting. Course: Use of QSAR models for REACH. E. Benfenati, E. Boriani and G. Gini.
2010 June 15th, Milan, Italy. TRISK course: “Identification and assessment of genotoxic and non-genotoxic carcinogens”. Lecture by E. Benfenati.
2010 December 15th, Milan, Italy. Course: “Metodi Alternativi alla Sperimentazione Animale in Tossicologia.”. Lesson by E. Benfenati.
2011 May 15th, Milan, Italy. SETAC Europe 21st Annual Meeting. Course: Use of QSAR models for REACH. E. Benfenati, A. Roncaglioni and G. Gini.
2011 July 4th – 8th, Feltre (Belluno), Italy. 2° Scuola Nazionale di Chimica dell’Ambiente e dei Beni Culturali. Il presente e il futuro. Lecture by E. Benfenati.
2011 August 23rd, Copenhagen, Denmark. RECETO PhD course: Use of QSAR models - practical use of the CAESAR models for legislative purposes (e.g. REACH legislation). Participation: E. Boriani, as teacher.
2011 September 9th, Padua, Italy. Master: "Strategic Environmental Management". Talk: Use of QSAR in environmental risk assessment. E. Boriani.
2011 November 7th, Istituto Mario Negri, Milan, Italy. Course on “Metodi Alternativi alla Sperimentazione Animale in Tossicologia”. E. Benfenati.
2011 December 2nd, Naples, Italy. Master di II livello in: REACH, Registration, Evaluation, Authorisation and restriction of Chemicals (CE n. 1907/2006). Participation: E. Benfenati and G. Gini as teachers.
2012 March 6th, Milan, Italy. Course: "Il metodo QSAR e le sue applicazioni pratiche nel Regolamento REACH". Participation: E. Benfenati.
2012 March 24, Bari, Italy. Master II livello in: "Regolamenti REACH (EC 1907/2006) e CLP (EC 1272/2008): valore alla sostenibilità dei processi produttivi e alla tutela della salute". Lesson: QSAR-REACH: strumenti operativi e scenari. A. Lombardo.
2012 May 18th, Milan, Italy. Doctorate program of "Politecnico di Milano". Lesson about Computational methods in toxicology. G. Gini.
2012 May 20th, Berlin, Germany. SETAC Europe 22nd Annual Meeting - 6th SETAC World Congress. Course: Use of QSAR models for risk assessment: practical use of the VEGA models. E. Benfenati, G. Gini and E. Boriani.
2012 August 22nd – 29th, Sao Paulo, Brasil. Course on QSAR. E. Benfenati and G. Gini as teachers.
2012 September 21st, Padua, Italy. Master di II livello in REACH (Venice and Padua Universities). E. Benfenati as teacher.
2012 October 12th – 13th, Naples, Italy. Master di secondo livello in REACH. Lecture: Metodo QSAR e sue applicazioni nel Regolamente REACH. E. Benfenati and G. Gini.
Seminars and Meetings at EU-Regulation Bodies
During the project period the ORCHESTRA partners designed and conducted five seminars with key stakeholders, including members of the main European and National Agencies.
The first two seminars were hosted by ECHA in June 2010 in Helsinki and EFSA in Parma in October 2010. In 2011 three seminars were planned with main European agencies EPAA, EMEA and EEA, which are associated with topics of ORCHESTRA. They were asked to host one workshop each. A seminar for the European Environment Agency (EEA) were organized and held on September 13th in Copenhagen 2011.
EPAA and EMEA were interested in the topic of the ORCHESTRA Project, but both didn’t agree to host a workshop due to time constraints and a lack of venue capacities. However, both institutions have been involved in other events and activities.
In the end the project partners decided to ask other key partners and stakeholders on the national level to host a workshop. The German Environmental agency (UBA) and the German institute for risk assessment (BfR) hosted one workshop each. Both do a lot of research activities with in silico topics and are well known in the European scientific community, and they turned out to be excellent partners, with a very constructive cooperation. The seminar hosted by UBA followed in November 2011, the seminar hosted by BfR in May 2012.
Furthermore, we organized in October 2011 a presentation of ORCHESTRA within the OECD workshop of the OECD group on in silico methods.
The goal of all seminars was to communicate and disseminate in silico methods and their potential use to regulatory agencies and improvement in the REACH context. The in-house seminars had the advantage that in a small group of people benefits and limitations of the methods could be discussed without any resentment. The participants got to know the needs and concerns from industry and developers/science, which the ORCHESTRA partners communicated to them. Together with the ORCHESTRA partners they worked out possible solutions how to cope with obstacles for the registration of substances. Additionally it was an important step to discuss and evaluate with the regulatory agencies the case-by-case policy.
Method of the Seminars
The main objective of the all seminars was to understand and discuss with regulators how one can use QSAR for environmental and human endpoints. The regulators have to approve chemical substances in the course of their work on basis of a registration dossier. Therefore it is very important to communicate and evaluate the potential use with them, discuss conditions of use and elaborate limitations. The participants should work out which information and data have to be provided for an approval and how the documented data should be interpreted.
Hence the main goals for all seminars were:
• to assess in silico methods as one methods to fulfil the REACH requirements from a regulatory view
• to realize the doubts, needs and concerns from regulation and industry and search for solutions how to cope with them
• to discuss successful cases and case by case policy and elaborate the uncertainty and limitations of the cases
• to collect feedback how in silico methods should be introduced in the scientific community of toxicology
At this level the focus of the debate was on experience and competence of the participants. This type of debate did not rely only on technical expertise, but gaining trustworthiness. Hence it was important to offer a format of the seminar in which the people could speak open without any resentment. As a method the partner decided to design a small seminar where the participants know each other (or get to know each other) and only the staff of the agency and the ORCHESTRA members participated. The seminar was conceptualized as an internal workshop. These agencies agreed to function as hosts, and the seminars were conducted at the premises of the hosts.
In most cases the partners divided the seminar in two sessions:
1. In one part ORCHESTRA members gave an overview of the project itself, presented the status quo of in silico methods and research or procedure and QSAR-models in regard to regulatory aspects. First ORCHESTRA members or the agency members presented followed by an open discussion round and questions and answer sessions.
2. The second part of the session was especially focused on case studies and demonstration of the website VEGA or other examples of in silico tested chemicals. The second part often followed a discussion in small groups about possible acceptance of the documented results.
Seminar at ECHA, Helsinki, Finland
The first seminar took place on 29th of June 2010 in Helsinki, Finland. It was hosted by the European agency ECHA. The seminar was split in two parts. In the morning session a small group of nine members and the ORCHESTRA partner joined the session. After a welcome and introduction from Doris Hirmann (ECHA), Emilio Benfenati (Istituto Mario Negri) and the other project partners presented the ORCHESTRA project and gave example of possible procedure. It was the idea to develop a way of good practice for in silico methods. The session ended up in an open discussion how to bring in silico method forward.
For the second part of the seminar the ECHA staffs was invited and around 25 members form ECHA participated. Emilio Benfenati presented the state of research on in silico method with the target to inform the staff about a usefulness if the method and to gain more acceptance.
The ECHA staff and the project partners discussed various issues and areas of cooperation:
• It provided basic information of in silico method for the ECHA staff and the opportunity to discuss several computational toxicology tools.
• It developed a first step of good practice of the use of In silico methods.
• Continue the two-way communication with ECHA as the main regulatory agency who evaluates the documentation of in silico methods.
Participants
The seminar was organized in two sessions. Fifteen people participated to the first session:
• Doris Hirmann and Rosella Demi, hosts and contact persons, ECHA, Helsinki
• Project team ORCHESTRA, Ludger Benighaus, Giuseppina Gini, Ralf Knauf, Dr Emilio Benfenati, Dr Simon Pardoe, Frank Lemke (for project ANTARES)
• Additional seven other members of the European Chemical Agency (ECHA)
Around 25 staff members ECHA from various departments and areas of interest participated to the second session.
Seminar at EFSA, Parma, Italy
The second seminar took place on the 27th of October 2010 in Parma, Italy. It was hosted by the European agency EFSA. The seminar was split in two parts.
In the first session member of the JRC presented the basics of QSAR and demonstrated the OECD QSAR toolbox “read across” and Toxtree. Then they gave it a try to test substances directly at the screen, with a software and database, followed by a broad discussion of the results and the acceptance. In the second part of the session Emilio Benfenati, Alessandra Roncaglioni (Istituto Mario Negri) and Giuseppina Gini (Politecnico di Milano) showed different results of CEASAR and OSIRIS project and presented the goals of ANTARES and ORCHESTRA. Then they demonstrated different test substances too. In the end there was a wide discussion how users could help to improve the use of models and get higher acceptance of the method.
Seminar at EEA, Copenhagen, Denmark
The third seminar took place on 13th of September 2011 in Copenhagen, Denmark. It was hosted by the European agency EEA and IRFMN and POLIMI participated.
The EEA staff and the project partners discussed various issues and areas of cooperation.
The main emphasis and interest was, of course, on the properties relevant for environmental properties and ecotoxicological effects. Indeed, most of the detailed information about environmental pollutants refer to the so called macro-pollutants, to heavy metals and inorganic compounds. The data available on organic compounds refers to a few main classes, such as pesticides, and solvents.
The availability of models for a larger set of organic pollutants is surely of interest.
Participants
• Staff from EEA
• Emilio BENFENATI, Rodolfo GONELLA DIAZA, Mario Negri Institute, Milano
• Giuseppina GINI, Politecnico di Milano
Seminar at UBA, Dessau, Germany
The seminar “REACH & QSAR Models for Environment – Possibilities & discussions on case studies” was held on November 22nd 2011 in Dessau-Roßlau, Germany, at the Federal Environment Agency (UBA). As the workshop was focused on regulatory aspects the participants were staff members of the UBA itself, mainly of the 4th Division - Chemical and Biological Safety.
The workshop was organized in two parts:
1. In the first part an overview was given via presentations about the project ORCHESTRA itself, QSAR-models in regard to regulatory aspects and the VEGA platform, which provides online access to several QSAR models for regulatory purposes;
2. The second part was focused on case studies and the discussion about the possible acceptance of the documented results. The detailed description of this part, due to its particular importance has been reported separately in section “Meet the German regulators: seminars and exercises” at page 24.
The main objective of the workshop was to understand and discuss with regulators who have to approve chemical substances in the course of their work on basis of a registration dossier:
• How one can use QSAR for environmental endpoints?
• Which information and data have to be provided for an approval?
• Where the limitations of this approach are and especially how the documented data should be interpreted?
Nine UBA representatives and five members of ORCHESTRA participated to the seminar.
Seminar at BfR, Berlin, Germany
The fourth workshop took place at BfR in Berlin on the 23rd of May 2012. During the first part of the workshop Ralf Knauf (Centro Reach S.r.l.) gave an overview about the project ORCHESTRA itself, while Emilio Benfenati (Istituto Mario Negri) and Giuseppina Gini (Politecnico di Milano) introduced QSAR models in regard to regulatory aspects and the VEGA platform, which provides online access to several QSAR models for regulatory purposes.
The workshop was open to a quite specialized public. The afternoon session was organized to have a more face-to-face discussion on mutagenicity. This group in the afternoon was slightly smaller than in the morning, with a more specific technical discussion.
The BfR staff and the project partners discussed various issues and areas of cooperation.
The BfR is surely interested on the possible use of some reliable QSAR models. The interest is on human endpoints, of course, such as mutagenicity, carcinogenicity, etc. Genotoxicity seems to be a good endpoint to be evaluated, and a PhD student is working on that.
The discussion in the field will be continued through further contacts. BfR was also interested in the results of the evaluation of the ANTARES project, testing a number of QSAR models.
Participants
• Staff from BfR
• Emilio BENFENATI, Mario Negri Institute, Milano
• Giuseppina GINI, Politecnico di Milano.
• Ralf KNAUF, Centro Reach, Milan
• Frank LEMKE, KnowledgeMiner, Berlin
Conclusions
The seminars at the premises of European and national agencies of member states turned out to be an excellent tool to discuss in small groups of experts the development and future acceptance of in silico methods. In the five seminars the ORCHESTRA partners introduced the basic information about in silico methods, QSAR models for environmental and human endpoints and the procedure of the model.
The seminars promoted the ORCHESTRA project and the results of different European project as ANTARES, OSIRIS, CEASAR or ANTARES, among others, which could be useful. They explained the case-by-case policy, listed successful cases, demonstrated the VEGA software and gave the opportunity to try out test substances at selected platforms and databases.
The participants discussed possible options, conditions and limitations of in silico methods and the steps for the next years. They listed the main concerns and problems with the applicability of the methods and how the data could be interpreted. They developed a way of good practice of in silico methods and thought about how to get a higher acceptance of the method.
Some problems were discussed for the acceptance of in silico methods. The following ones were often communicated and are the main conclusion if in silico methods are accepted by regulatory agencies in the future.
One main problem of in silico method is that scientists use in silico methods and QSAR very often, but not in the right way. They take only the data, which suits them, and not really proof if other data brings out different results. The regulatory agencies have to proof this procedure and are not allowed to rate on the basis of such data e.g. in PBT screening.
Another issue, which was often communicated, is that the users in regulatory agencies can't really deal with estimations. They have to accept a value as correct/not correct and should be reminded when promoting QSARS. QSAR can always give an idea of the toxicity of “endpoint-picture”. One discussed opportunity was to compare results with different models.
An additional point was that automatically created decisions of in silico tools are often a kind of a black box. In silico methods should try to keep the procedure of the assessment transparent and reliable. The regulatory agencies are often missing the documentation of the combined information. They do not need only the prediction by the model but also the read across and the combined information to understand what was done.
But also for the user it is important that he/she knows what the theory of the model, information about descriptors and what the compounds of the model are. They want to understand for example why some compounds are “out of domain” or not and what similar compounds do exist which they can use.
The system and platform VEGA however provides a good balance between automatically received information and the expert and user decision. The VEGA website gives the opportunity to think about the outcome of the model results and interpret them. It doesn´t force the user to use them without any proof. ORCHESTRA try to incorporate more models than one and put out information about the descriptors.
One crucial point is to find out if the model is applicable for the chemicals, which are to be tested. Therefore the main important thing of a model or software is that it asks the right questions to identify the right model AND answers them in a really transparent way.
The guidance and the training of the user is a very important step in getting the model more applicable.
Finally it does make sense that it is better not to promote one model only and be open to other one. ORCHESTRA should promote the idea that it is possible to use the prediction of QSAR models and there are some pieces of information that creates more confidence in regard to the evaluation of the results.
Delphi Experts Workshop
One important milestone of the project was the Group-Delphi-workshop on “Successful Communication of Scientific Content on the Example of Alternative Testing Methods of Chemical Substances” which held on December 13th, 2010 in Stuttgart Germany. Sixteen experts from chemical industry, NGOs, communication and public relations, as well as science communication practitioners participated and all gave valuable evaluation and suggestions about in silico methods.
Objectives and Agenda
The workshop aimed at the examination of communication strategies on the basis of expert judgments with the objective to promote a wider understanding, the awareness and acceptance of in silico methods. For this purpose various concepts and approaches to solution strategies among the scientific community were discussed. Thereby was the overall goal to strengthen and expand the dialogue in the scientific community.
Guiding Questions of the Expert-Delphi
The participants worked on a comprehensive 20 pages questionnaire during the full day workshop, which follows specific guiding questions
Structure and methods
The Group Delphi method used in this project is a variation of the classic Delphi method and is appropriately designed to find out the common ground among the experts, the reasons and arguments that cause differences in assessments. During a group Delphi all participants meet face to face and make the assessments in randomly assigned small groups of three or four. The groups whose average scores deviate most from the median of all other groups are requested to defend their position in a plenary session. Then the small groups are reshuffled and perform the same task again. This process can be iterated two or three times until no further significant changes are made. At the end of a Delphi process, one receives either a normal distribution of assessments around a common median, a two- or three-peak distribution (signalling a majority and one or more minority votes) or a flat curve (which means that knowledge is insufficient to make any reliable assessment).
Workshop Outcome - Recommendations to ORCHESTRA
From the results of the group Delphi, on the basis of expert judgments, the following excerpt of recommendations for the communication of in silico methods with stakeholders, regulators and the public have been derived. The full report can be downloaded in English or German versions on the ORCHESTRA website:
http://www.in-silico-methods.eu/sites/default/files/documents/pub/group-delphi-communication-orchestra-results-english-version.pdf
http://www.in-silico-methods.eu/sites/default/files/documents/pub/group-delphi-communication-orchestra-results-german-version.pdf
In silico methods and their use
The participants assess the use of in silico methods still to be low. The crucial point is how the perception and acceptance of in silico methods as alternative or combined testing strategy can be enhanced. In addition to the recognition of the methods in the REACH process that envisaged an extensive documentation, the experts see the development of new methods like in silico as expansive and time consuming.
However, the recognition of in silico methods by regulatory authorities could lead to a considerably reduction of costs and efforts of the alternative testing methods. It is important that besides a dialogue-focused way of communication it should be made use of one-way-communication strategies that focus on the communication of (pure) information as well. Instead of a broad PR campaign the experts recommend a targeted PR-strategy that addresses primarily specialized journalists. According to the experts in silico methods are in general very specific. Therefore the media should play a minor role in favour of the direct communication with stakeholders and regulators. Rather than isolated in silico testing methods should be reported in the context of animal testing and experiments with cell cultures to increase the response of the media.
Dialogue with Stakeholders and Regulators
In addition to pure information an intense dialogue on the advantages and disadvantages, necessary conditions and extensive use and application potential is advisable. The ultimate objective of the dialogue is to provide information to stakeholders and regulators so that they can coordinate policy making, economic and political action with regard to the knowledge provided. Thereby, a well-structured and organized discourse among experts in which as many as possible stakeholders and researcher are involved in is very important. In order to promote the professional exchange and trust, small discussion groups and workshops are appropriate communication formats.
A major potential to bring actors the complex issues of in silico methods closer lies in specific courses and trainings. The information should be presented in a way that enables stakeholder and regulators to form their own opinion.
Dialogue with the public
Overall, the reciprocal exchange and dialogue with the public should be enhanced to order to achieve a wider public understanding of chemical testing methods. Due to prior knowledge of the citizens individual offers of information (e.g. brochures and press releases for comprehensive information as well as more in-depth information) to strengthen the opinion building process.
A free resource of in silico models: the VEGA platform
VEGA (Virtual models for property Evaluation of chemicals within a Global Architecture) is a new freely usable on line platform for the prediction of physicochemical, (eco)toxicological and environmental properties.
The VEGA platform can be used through its official website (http://www.vega-qsar.eu/).
VEGA has been developed by Istituto Mario Negri (leading partner of ORCHESTRA) in collaboration with U.S. Environmental Protection Agency (US EPA), Politecnico di Milano (Italy), The Food and Environment Research Agency (FERA, United Kingdom) and KnowledgeMiner Software. This new platform is also supported by several European projects (CAESAR, ANTARES and ORCHESTRA) and Italian authorities (Ministry of Health, Ministry of Environment and “Istituto Superiore per la Protezione e la Ricerca Ambientale”).
The aims
The main aims of VEGA are clearly stated in the homepage:
• Our Vision – To enhance the in silico capabilities to assess chemicals.
We want to sustain the correct use of in silico/QSAR models, as a way to support the evaluation of the chemical safety. We want to provide computer tools not competitive to the human experts, but providing support to the experts.
• Our System – To make freely available in silico models, on line and in a downloadable version, together with explanation on the correct, transparent use of the software.
The availability of free methods will promote their use and debate, improving their reliability.
• Our Community – To aggregate laboratories, experts, sponsors, sharing the interest for in silico methods useful to reduce the risk of chemicals.
There are many valuable initiatives producing computer methods capable to assist in the evaluation of chemical risks. Our community provides a network of actors active in this. We solicit contributions in this direction.
• Our Mission – To reduce the impact of chemicals on human beings and environment.
There is a huge need of safer chemicals, and of a better knowledge on the effects of chemicals, both natural and man-made. We should identify the most risky ones, to concentrate our efforts on the reduction of their adverse effects. In silico methods can be very useful, if correctly applied.
Improving the reliability of results
The Applicability Domain and Reasoning information provided by VEGA are aimed to help understanding the reliability of the predictions obtained by the enclosed QSAR models.
Applicability Domain
The Applicability Domain (AD) tools, using several parameters such as similarity with molecules present in the model database, how the model performs on these molecules, etc., is able to define if submitted molecules can be predicted by the model or not.
A novel parameter called Applicability Domain Index has been introduced. For each submitted molecule, VEGA computes the AD-related parameters and combines them to obtain the AD index, which ranges from 0 to 1; higher ADI values mean reliable predictions whereas lower values mean unreliable ones.
Using AD information, in one way seems to limit the number of molecules which can be predicted, but on the other hand increases the reliability of those which fall within the model’s AD. Having the safety of chemicals in mind, this feature results of high importance.
Together with AD, VEGA also provides information about the six most similar compounds which are present in the model database. Particularly, for each similar molecule the structure, experimental and predicted values are shown.
This feature can help experts in performing a sort of read-across. On the basis of data provided, the user could, for example, decide if the prediction obtained for the submitted compound is over- or underestimated.
Applicability Domain Index and Reliability
Within EC Project ANTARES, performances of several available QSAR models have been assessed. Where available, information about applicability domain have been also utilised to check how different the models behave with model inside and outside their AD.
Considering as example the VEGA model for bioconcentration factor (BCF), within ANTARES a dataset of 860 molecules with known experimental values has been used to check the model performances.
The application of higher ADI thresholds lead to consider less prediction as reliable; the mean error is however lower, thus increasing the reliability of molecules considered as “predictable by the model”.
Reasoning
The other new and useful feature introduced by VEGA is reasoning. The main aim in this case is to explain the user why a molecule has been predicted in a certain way.

Potential Impact:
Impact on the future dissemination and exploitation of EU projects

In order to help research projects dealing with in silico methods in improving the efficiency of the dissemination of their results, ORCHESTRA initially studied what had been done by other projects (selected as case studies) and gained information about barriers and needs, directly from stakeholders (such as members of regulatory bodies and chemical industry).
The information, suggestions and observation obtained have been helpful in the collaboration with the project ANTARES, which started on January 1st 2010, for the development of the VEGA platform. Thus, we can say that this novel tool has been designed in a “by users for users’ needs” way. ORCHESTRA analysed tools produced within the selected EC funded projects (mainly CAESAR and DEMETRA), what they offer and their characteristics. The results of this analysis were combined with the opinions collected from users using several channels, such as questionnaires, interviews and courses on the use of in silico models. From the resulting information, ORCHESTRA elaborated suggestions for the development of VEGA, which could be of interest within future EC funded projects.
Furthermore, ORCHESTRA analysed the dissemination strategies of past EC projects and gathered information on barriers and opportunities related to the use of in silico methods in toxicology for regulatory purposes. The information collected were utilized to develop a dissemination plan to use within ORCHESTRA and that could be of interest for future projects. ORCHESTRA itself acted to test dissemination strategies, initially promoting models developed by DEMETRA and CAESAR projects and the early beta releases of VEGA. The dissemination and promotion focus then shifted, thanks to the collaboration with ANTARES, of the VEGA platform. Several dissemination activities were established, targeting different stakeholders (e.g. the seminars at regulatory bodies’ headquarters and the booth at SETAC 2011, which targeted regulators, scientists and representative of chemical industry).
Apart from the direct dissemination of in silico models, ORCHESTRA developed informative materials about in silico methods and their possible use for regulatory purposes. This material includes several video documentary (containing, among other things, interviews with regulators and other stakeholders), scientific articles, the introductory leaflet, the ORCHESTRA e-book and the layman report. All these resources have been made available through the ORCHESTRA web portal and could represent an important starting point for future projects, which could utilize the opinions and suggestions collected, to develop tools and resources more useful and interesting for stakeholders.


Impact on the regulators

ORCHESTRA organized several meetings and workshops with regulatory bodies in Europe; members of the project also actively participated to international events, where meet and discuss issues and advantages of in silico methods, with regulators.
Within these numerous contacts, we established good contacts with certain national regulatory bodies. This refers mainly to groups within Germany (BfR and UBA) and Italy.
In Italy the interest was very high, in particular in relation with the possible use of in silico methods within REACH. The national authority for REACH, Ministero della Salute, promoted the VEGA web site (see below). Support has been obtained also by The Italian Ministero dell’Ambiente e ISPRA, which is the research institute on environmental issues, supporting the Ministero dell’Ambiente.
Furthermore, ORCHESTRA promoted the idea, at the Italian level, of a Group of interest on QSAR models, supported by the three Miniteri della Salute, dell’Ambiente and Sviluppo Economico. Indeed, Health, Environment and Industry authorities are the three key actors for REACH in Italy.
These three authorities established this Group of interest, which is coordinated by Emilio Benfenati, the coordinator of ORCHESTRA, Members of the group are main Universities and research centers in Italy, including Prof. Gini, of Politecnico di Milano, another partner of ORCHESTRA. This shows a very important impact of the project, which convinced national authorities to support the in silico models for REACH. The Italian Group is expected to make predictions for industries in Italy, for REACH.
VEGA and, more in general, in silico methods, received positive feedbacks from regulators during these and the other events. They recognize the usefulness of these methods as integration to other results (e.g. from in vitro tests).
Relying on these comments and the suggestions received during these and the other events, from on line exercises and from the statistics on the download of VEGA, we are confident that ORCHESTRA successfully attracted people from regulatory bodies to use computer based models for the assessment of chemical properties for regulatory purposes.


Impact on international harmonization

ORCHESTRA participated to the 6th meeting of the OECD QSAR Toolbox Management Group (October 26th – 27th 2011, Berlin, Germany) with a speak by Emilio Benfenati (Istituto Mario Negri), presenting results of the project and the VEGA platform. Moreover the OECD QSAR Toolbox were presented during events organized by ORCHESTRA, such as the seminar at EFSA (October 27th 2010 , Parma, Italy) and the ORCHESTRA workshop (April 6th 2011, in Milan, Italy).
Many participants were very positive on the approach taken by ORCHESTRA and appreciated the VEGA platform, for its success in being helpful and explicative for the user, within an user-friendly scheme. One of the advantages of VEGA is to contribute to make explicit, and thus objective, an evaluation on the result of the QSAR models, which in many cases is subjective. This is of great help for the harmonization of the evaluation of the results of QSAR models.
Moreover, ORCHESTRA organized exercise and surveys including several in silico tools (VEGA, OECD toolbox, US EPA T.E.S.T. and US EPA EPI suite) so that stakeholders could learn about available in silico resources and compare them and their results.
Also this exercise, the only of this nature, contributed to start a discussion on the evaluation of the results of QSAR models, again in the direction of harmonization of the assessment.
The outcomes obtained from these initiatives were summarized and commented by ORCHESTRA partners and, in some cases, by the developers of the software. ORCHESTRA presented these results with posters and talks at several international events; two articles have been also written and published:
- Benfenati E., Gonella Diaza R., Cassano A., Pardoe S., Gini G., Mays C., Knauf R. and Benighaus L., “The acceptance of in silico models for REACH. Requirements, barriers, and perspectives”, Chemistry Central Journal 2011, 5:58.
- Benfenati E., Pardoe S., Martin T., Gonella Diaza R., et al. (Forthcoming), “Using toxicological evidence from QSAR models in practice”.


Impact on industry

We already mentioned the Italian initiative to establish the national Group on QSAR. This group established an internet web site (http://www.smart-reach.eu) which is dedicated to provide support to industries for REACH, offering upon payment predictions for the chemical substances to be evaluated.
Italy has a particularly high amount of SME in the chemical sector, which makes week this industrial sector, because the costs for the dossiers are quite high for SME. Furthermore, these SMEs have a large number of chemical substances on the market at relatively low tonnage. An example is the dye industry, with a high number of SMEs, and many hundreds of substances to be eventually registered. The risk is that many substances will not be registered, and many SMEs will not survive.
In silico models can mitigate this situation.
ORCHESTRA collaborated with other EC projects (e.g. ANTARES), software developers and national regulatory bodies in the development of the VEGA platform. This software have been designed specifically for regulators’ and chemical industry’s needs in the current regulatory framework (such as REACH).
The REACH regulation requires industries to deal personally with the evaluation of several properties (including toxicological and ecotoxicological) of chemicals. Comply with such duties by mean of classical animal testing, would imply very high costs (often out of small enterprises’ possibilities), long times and the use of a number of vertebrates beyond the acceptable by the public opinion.
ORCHESTRA promoted in silico methods as a possible solution for avoid high costs, long times and use of too many animals (therefore complying with the 3Rs principle). The VEGA software was presented to industrial representatives during international events, courses organized specifically for them and news published for example on newsletters of the Italian National Federation of the Chemical Industry (FEDERCHIMICA).
As for regulators, VEGA offers industries a tool useful to evaluate several physico-chemical, (eco)toxicological and environmental properties. VEGA, as all in in silico tools in general, is faster than both in vitro and in vivo tests and can evaluate thousands of compounds in few minutes (as checked within the ANTARES project). Moreover, this tool has been developed as a freeware, thus reducing significantly the costs of evaluation.
Results obtained with VEGA are suitable be used in several ways:
- For the compilation of the REACH-requested dossiers, even better if utilized with results from other tests in a weight of evidence approach;
- For prioritization, using the prediction to create a list of the most concerning chemicals which should be tested with other methods;
- In a pro-active approach, during the design of new compounds, avoiding the high costs of the development of possible toxic compounds.


Impact on chemical safety for environmental protection

As already explained in the previous section, in silico methods represent a fast and cheap way to evaluate the toxicity of chemicals, both towards human and the environment. This evaluation can be done on existent chemicals or in the early steps of new chemicals deign. Thus, in silico methods can be of high importance in the improvement of chemical safety.
Moreover, apart from predictive models, other important computer resources exists: the databases. Collections of chemical properties (including toxicity) and structures have been developed for years, several of them have been made freely available as on line databases.
The ORCHESTRA web portal, being developed to become a central hub of information for in silico methods, reports and promotes the most relevant and freely usable database available. These resources have been grouped, on the basis of the type of data included.
During talks and presentation given at events as well as while teaching in silico methods at organized courses, ORCHESTRA members underlined the current and future the importance of the combined use of databases and predictive models. Several initiatives exists at international level, aimed at producing data on chemical properties using for example in vitro testing approaches. This means that in the next few years a huge number of toxicity data will be available and that in silico methods would play a key role in their interpretation.


Impact on techniques alternative to animal models

The number of animals used annually in Europe for tests is about 10 million and only a small percentage of this figure will be those to be used for REACH.
ORCHESTRA embraced the so-called 3Rs principle, which state for Replacement, Refinement and Reduction of animal testing.
As already explained in the previous sections, in silico methods could play a key role in the progressive decrease of the number of animals used.
At the current state, in silico methods are not able to completely replace the other approaches. For this reason, the term integrative methods has become preferable instead of alternative methods. The question is:
“how to make the novelty of in silico approaches acceptable and usable by experts who have used other and more established methods for years, in order to integrate all possible data with the aim of improving the chemical safety?”
Several initiatives and projects have been funded in Europe with the aim of integration of different approaches. Three of the projects selected by ORCHESTRA as case studies dealt with this argument:
- RAINBOW was a one-year project funded specifically to organize a workshop on the integration of in vivo, in vitro and in silico methods. The workshop, entitled “The RAINBOW workshop on the integration of in vivo and in vitro data with computer-based modelling” was then organized on December 11th – 13th 2006 at Istituto Mario Negri in Milan (Italy);
- OSIRIS has been probably one of the most important example of integration between different approaches; its main aim was to develop the so-called “integrated testing strategies” (ITS) for endpoints relevant in REACH. An ITS, in few words, is a sort of flow chart which aims at drive the assessor in a step-by-step check of the information available and which also suggests the most appropriate tests and methods to obtain missing data.
- CASCADE was a 5 years Network Of Excellence project, which mainly dealt with food safety. In silico methods were not the key approaches utilized within this project, but anyhow played an important role as integration to in vitro tests. Within CASCADE 3D-QSAR models have been developed as well as docking models for virtual screening of libraries of endocrine disruptors against nuclear receptors (e.g. estrogen receptor). These models were then coupled with laboratory tests to asses, for example, the toxicity of metabolites.


Impact on in silico modelling. A higher quality

ORCHESTRA actively collaborated in the development and promotion of the VEGA platform, which includes several models for the prediction of physico-chemical, (eco)toxicological and environmental properties relevant mostly for REACH regulation. In this context, the aim and efforts were not in the development of models themselves; the models included in VEGA have been provided by other EC projects (e.g. CAESAR) or agencies (e.g. US EPA).
The main aim of ORCHESTRA was to collaborate with stakeholders, such as Chemical Industry and Regulatory Bodies, in order to better understand stakeholders’ needs and barriers in the use of in silico methods. The acquired knowledge has then been used for the fine tuning of VEGA. ORCHESTRA’s efforts has been mainly aimed to the development of tools able to improve the reliability of QSAR predictions, such as the Applicability Domain (AD) evaluation.
Both the REACH regulation and OECD list the applicability domain issue within the requirements defined for the use of QSAR predictions.
According to OECD:
QSAR results can be used instead of tests when the following conditions are fulfilled:
• …
• The chemical belongs to the QSAR model’s regulatory area of influence (the substance falls within the applicability domain of the QSAR model);
• …
According to REACH:
Results obtained from valid qualitative or quantitative structure-activity relationship models QSARs may indicate the presence or absence of a certain dangerous property. Results of QSARs may be used instead of testing when the following conditions are met:
• …
• the substance falls within the applicability domain of the QSAR model,
• …
ORCHESTRA presented VEGA, its features and how it provides the results, during seminars and international events and gained feed backs from stakeholders on how to improve the software for regulatory needs.


Impact on the European research

ORCHESTRA presented a good opportunity to discuss the exploitation strategy for the European research.
A specific deliverable of ORCHESTRA summarized the key lessons which have been derived from ORCHESTRA, and can be useful for a broader, more general situation.
The dissemination and the exploitation of the results of the EC projects are too often considered as a task successive the project, which represent a formal duty. On the contrary, the application of the results should be considered a fundamental part of the research itself, representing the proof of principle of the correctness of the approach. This kind of activity should proceed along the project development, and be part of it as soon as possible, in order to incorporate the necessary elements arising from the potential application, as inspiring issues, leading the project towards success. Exploitation and practical application quite often will have an impact also on the theoretical strategy applied within the project, thus it would be wrong to think that the exploitation is only a pragmatic issue.
A series of key aspects in this evaluation are presented and discussed within the results of ORCHESTRA, Deliverable 6.3.
Impact on developing countries
The global production, trade and use of chemicals have increased significantly in the last years. This fact is placing more burdens for the management of chemicals in developing countries and in those with economies in transition. This new challenge presents special difficulties for these countries, which are placed in the position of not being able to meet the requirements.
The potential impact of in silico predictive tools on these realities, mainly due to their lower cost compared to the other methods, is very high. ORCHESTRA established several mail contacts with scientists India, Thailand, Korea, Brasil and Argentina.
Moreover, members of the project were invited in August 2012 in Brazil, where they participated in the introduction of QSAR methods in this country through the first in silico course offered in the Sao Paulo University, the first ranked University in Brazil.
These countries are particularly active. Brazil is an outstanding example. This country, and many others, are very dynamic. In silico models are particularly suited to be adopted by these countries. During our visit to Brasil, Sao Paulo and Campinas, we found universities and industries particularly interested in these tools. We transmitted the knowledge on how to use a series of tools. The application to some case studies of interest for the local pharmaceutical industries was immediate and successful, and a joint paper was submitted. This demonstrated the easy and immediate possibility to apply these tools, thanks for their easiness and low cost, and lack of need of hardware and expensive technology.
Impact on European research dissemination practice
The 2008 call indicated that the projects should be viewed as ‘case studies’ from which further insight on research dissemination could be developed. A final task was to review the experience of dissemination gained in the ORCHESTRA project. Viewing the project as a case study, we identify some of the challenges, practical strategies, successes and lessons from the project which have wider application in the dissemination of EU environment, science and technology research.
The outputs is an updatable e-guide, with the title ‘Disseminating science and technology research: a practical e-guide for researchers’. It is intended to complement what is already available from the EC and other research funders. It focuses on the practical and intellectual challenges of research dissemination, and identifies priorities and strategies.
It is available online and will be updated from further work. It will also form part of a forthcoming publication for researchers, on the practical and intellectual challenges and priorities in research dissemination. As it is not about in silico methods, it is not available from the ORCHESTRA website, but from: www.researchdissemination.eu/guide

List of Websites:
The ORCHESTRA web portal

The ORCHESTRA official website was developed very soon at the beginning of the project, and its contents kept updated during the whole project life and beyond. This needed a change of the content of the web site several times.
In order to improve the accessibility, the web portal has been made available through three addresses:
- http://www.orchestra-qsar.eu/
- http://www.in-silico-methods.eu/
- http://www.insilicomethods.eu/

While the first one focuses on the project, the other two were chosen in order to clarify the aim of the portal to become a sort of hub for in silico methods.


Contacts

The project was coordinated by Dr Emilio Benfenati, head of the “Laboratory of Environmental Chemistry and Toxicology” at Istituto di Ricerche Farmacologiche “Mario Negri” in Milan (Italy).

Contact details
Address: Via La Masa 19, 20156 Milano, Italy
Telephone: [phone]
Fax: [phone]
E-mail: [e-mail]