Final Report Summary - 3GB-TEST (Introducing diagnostic applications of ‘3Gb-testing’ in human genetics)
Executive Summary:
In a two year EU funded (FP7) coordination support action (CSA) project: 3Gb-TEST, we explored the current status of whole genome sequencing to identify gaps and needs for its implementation in routine diagnostics. As history learned, novel genetic technologies, e.g. the many PCR based screening methods, often got implemented instantaneously, while proper validation only followed in performing the “diagnostic” tests. These blurring lines between research and diagnostics are usually caused by the limited number of samples in case of rare disorders. With all recent NGS (next generation sequencing) approaches a similar trend was observed, the myriad of equipment and available capture kits, targeted gene panels up to whole exome kits are being introduced instantly as diagnostic tools all with limited in house validation. Sequencing the total human genome (WGS), or a 3Gb-Test as we coined it, is coming close (probably within 5 years) to be implemented as a tool in healthcare. Therefore, our effort is early enough to explore and identify what needs to be in place before implementation. Workshops and expert meetings were organized to address the following topics:
-Wet lab innovations
-Bioinformatics’ tools
-Indications for WGS and its clinical utility
-Training of professionals
-Ethics
-Health technology assessment.
The outcome of 3Gb-TEST project is an extensive technology roadmap document and a guideline in which gaps and/or needs are specified. Amongst others the following major gaps or needs were identified:
-Lack of knowledge to interpret the non-coding regions of the genome
-Clinical, diagnostic trustworthiness of prediction and assessment tools is needed
-Sharing of genotype and phenotype information into the routine diagnostic practice is not in yet place
-Training laboratory and clinical staff to cope WGS data is needed
-Capacity building within Europe
-Regular update of NGS guidelines towards WGS is needed
Furthermore, it is critical that provisional recommendations on ethics are made to ensure that regulatory and expert bodies are able to develop standards. Use of relevant past experience in other domains could be utilized in order to be able to formulate such recommendations in the interim data gathering period.
Concluding, the overall consensus for the short term is that WGS is not yet ready for diagnostic use, but soon will be. We suggest to first expand the current WES analysis with CNV detection, by use of the novel WES capture kits with the capacity to also interpret genomic CNVs. This will replace Array analysis as a first line tool and is for the time being be the best (cost efficient) option. Further, if the price allows WGS could be used to read out the exome only and all genomic CNVs. This latter option would be an ideal way to bridge the gap between WES and full WGS, and add info to build up statistical (genotype) data on the non-coding regions of the genome.
Project Context and Objectives:
Introducing diagnostic applications of ‘3Gb-Testing’ in human genetics: What is still needed?
3Gb-TEST, a two year EU funded (FP7) coordination support action (CSA) project was set up to investigate the current status of whole genome sequencing to identify gaps and needs for its implementation in routine diagnostics. History has learned that novel genetic technologies, e.g. the many PCR based screening methods, often got implemented almost instantaneously, while validation followed in performing “diagnostic” tests. Blurring lines between research and diagnostics are usually caused by the limited number of samples in case of rare disorders. Similarly with all new NGS (next generation sequencing) approaches we noticed the same trends, equipment and many capture kits, targeted gene panels up to whole exome kits have been introduced almost instantly as diagnostic tests with limited in house validation.
Sequencing the total human genome (WGS), or a 3Gb-Test as we coined it, is close to be implemented as a tool in healthcare. Many ongoing research initiatives now evidence it’s putative strength. Sequence technologies are improving, getting faster and cheaper in generating large data sets. Bioinformatics’ tools perform increasingly better in detecting not only single nucleotide variations and insertion deletion variants (SNPs/INDELs), but also in detecting copy number variations and structural variants (CNVs/ SV’s). Although care has to be taken because none these “often in house” tools seem to be optimal yet. The same holds for variant effect predictor tools, variation between the different approaches can still be substantial, so that the only hard evidence can be found in wet lab ( in vitro or in vivo ) studies to proof the extend of the functional defects.
Since, fast and often sub-optimal implementation of novel technologies have been more rule than exception over the last few decades we hope to set the stage for WGS with this 3Gb-TEST project by defining what is missing and or needed while preparing a roadmap and guideline for its implementation.
With a total of eleven partners and embedded in a well-established network of collaborators the 3Gb-TEST project set out to address the following 7 objectives:
I. Exchange of experience and knowledge
II. Assessment of research resources, gaps, and other needs
III. Determine suitable applications for the new technology
IV. Health Technology Assessment (HTA):
V. The organization of an External Quality Assessment (EQA) scheme
VI. The organization of courses/symposia in the participant’s regions
VII. The writing of one or more guideline documents and a technology roadmap
Project Results:
With a total of eleven partners and embedded in a well-established network of collaborators the 3Gb-TEST project addressed the 7 objectives as follows:
I. Exchange of experience and knowledge: 3Gb-TEST has organized expert meetings two of which attached to the major international congress in our field (ESHG), exchange of views and insights helped to assess the current state of NGS and WGS in particular. Novel developments in the field were followed closely and showed us that research is moving fast but on the other hand that at many levels, equipment, bioinformatics software, and interpretation lots of variation still exists which urges to be cautious with the implementation of WGS.
Also the workshops and expert meetings permitted rich multi-disciplinary discussion around the complex ethical issues related to the clinical use of new sequencing technologies. Guidelines and experience are starting to accumulate, and it is envisaged that over the next few years further experience will allow guidelines to be honed and improved to better serve patients. For the moment some caution was urged in view of the difficulties of interpretation of results, and it was generally (but not unanimously) felt that with the present state of knowledge, opportunistic screening was not something that should be offered to all who undergo WGS testing
II. Assessment of research resources, gaps, and other needs: Wide range of Bioinformatics research are tools available, we catalogued around 180 in-silico bioinformatics tools. Overall outcomes for variant sets did not differ between different algorithms, but concordance is not complete and algorithms cannot be trusted as a single source of evidence. The involvement of phenotype information could improve the outcome of clinical-grade algorithms. Also a clear need for clinical grade variant databases to complement algorithms with available knowledge, also at phenotype level is detected.
III. Determine suitable applications for the new technology: Trends that have been identified over the last 5 years: NGS has been successfully implemented in the mainstream diagnostics, this after settling on European guidelines for NGS (Matthijs et al. 2015). Several successful approaches for different purposes can be mentioned, NIPT for prenatal numerical chromosomal aberrations, whole exome sequencing (WES) or panel-based NGS solutions for mutation detection in genetically heterogeneous conditions, depending on the required balance between high diagnostic yield and high analytical sensitivity, ampli-seq-based sequencing or multiple inversion probe (MIP)-based sequencing for ultra-deep sequencing applications, e.g. in molecular tumor diagnostics, or detection of somatic mosaicism. Soon whole genome sequence analysis (3Gb-testing) will be an effective and financially viable alternative to all targeted gene analysis approaches and could have an enormous impact on diagnostic centers replacing many existing molecular and cytogenetic tests. However, although the genome will provide all available information SNPs/INDELs and CNVs/SVs, the majority of the additional information from full genome sequence will be difficult to interpret. Because the normal variation is not known yet. So for the short term using novel WES capture kits with the capacity to also interpret exomic CNVs, in necessary followed by an array, might be the best option. Alternatively if the price allows WGS could be used to read out the exome only and all genomic CNVs. This latter option would be an ideal bridge between the WES an WGS and add info build up statistical data on the non-coding regions of the genome.
IV. Health Technology Assessment (HTA): A total of 16 systematic reviews were identified and summarised. A total of 55 model-based economic evaluations were identified. In general, resource use considerations were consistent. Most economic evaluations looked at resources involved with the test, staff, hospitalisation, and other medical procedures. Sources for test costs included re-imbursement rates, expert elicitation, laboratory charges and quotes from the technology’s manufacturer. No studies had precisely estimated costs using methods such as micro-costing. Economic evaluation should form an integral part of the HTA process because decisions to approve genetic tests for use in any EU member state has implications for finite healthcare budgets.
V. The organization of an External Quality Assessment (EQA) scheme: With the widespread adoption of Next Generation Sequencing (NGS) in diagnostic testing, the challenges for EQA scheme providers are that there is still little consensus on how to do diagnostic testing with an NGS based approach. Some labs have opted to replace Sanger sequencing based tests on small panels of genes, whilst others are implementing clinically relevant exome (WES) or whole genome (WGS) testing. In addition, many labs are still transitioning from using the technology in research to clinical diagnostics. The technology itself is also developing at a rapid rate with 2nd generation technologies already in widespread use, and the development of 3rd generation approaches nearing completion. With such variation, the challenge for EQA providers is to develop an EQA scheme that is generic, platform-agnostic and testing context independent whilst at the same time fitting seamlessly in to a laboratories workflow. Given the challenges outlined above, development of 3Gb-EQA schemes has been slow and currently there are only two EQA providers offering this type of activity; the College of American Pathologists (CAP) and EMQN. Both schemes have evolved independently and are method based EQAs applicable to labs using gene panels, WES or WGS but differ in the scope of testing required. Also the data received from the bio-informaticians varied because VCF file format is not a real standard format. So the first NGS EQA studies were varying in results and difficult to interpret.
VI. The organization of courses/symposia in the participant’s regions: 3Gb-TEST has organized 13 public symposia, courses, and workshops to exchange knowledge and ideas. These meetings were organized in eight different countries, the Netherlands, Germany, Italy, Greece, France, Sweden, the Czech Republic and the UK. In addition, members of the Consortium presented at other meetings, such as the 13th International Symposium on Mutation in the Genome in April 2015, which was co-sponsored by 3Gb-TEST. There were several scientific publications in peer-reviewed journals.
VII. The writing of one or more guideline documents and a technology roadmap:
Written information to the community, such as guideline document; Introducing diagnostic applications of ‘3Gb-Testing’ in human genetics: What is still needed? and a technology Roadmap have been produced. All data inclusive all other documents are available on the project web site: www.3Gb-TEST.eu. Due to the special design of the web site and the brochure, there was a lot of (enthusiastic) feedback from the scientific community.
Potential Impact:
With this 3Gb-TEST project we provided a forecast on the NGS/WGS developments. The future of molecular testing will pose a challenge to both laboratory and clinical geneticists, due to the wealth and complexity of the sequence information obtained. Clinical utility and anticipated difficulties of whole genome sequencing are discussed, as it is important that patients and families receive correct advice and appropriate management. It is also important to preclude mistakes with respect to ethics, quality issues, and over- or misinterpretation of data. Therefore, apart from technical issues and quality issues also ELSI (ethical, Leagal and Scocietal impact) issues were discussed with equal priortity.
Amongst others the following major gaps and needs were identified: 1. Lack of knowledge to interpret the non-coding regions of the genome; 2. Clinical, diagnostic trustworthiness of prediction and assessment tools which are an issue rather than the availability and range of options; 3. Too many factors are still in the way for introducing the sharing of genotype and phenotype information into the routine diagnostic practice; 4. Training laboratory and clinical staff to cope WGS data; 5. Capacity building within Europe; 6. Update of NGS guidelines towards WGS; 7. It is critical that provisional recommendations are made to ensure that regulatory and expert bodies are able to set standards and so the suggestion was made that relevant past experience in other domains could be utilized in order to be able to formulate such recommendations in the interim data gathering period.
The overall consensus for the short term is to first use the novel WES capture kits with the capacity to also interpret genomic CNVs, if necessary followed by an array analysis. As this might for the time being be the best (cost efficient) option. Further, if the price allows WGS could be used to read out the exome only and all genomic CNVs. This latter option would be an ideal way to bridge the gap between WES an WGS and add info to build up statistical (genotype) data on the non-coding regions of the genome. Meanwhile updating the various WGS guidelines is essential.
As, current gaps in our knowledge have been identified and research will to be initiated to bridge these gaps. The 3Gb-TEST project broad stakeholders together and ensure they are informed with respect to the desirable and undesirable developments. Furthermore an new initiative has been under taken to follow up this project with a Cost Action project (Target-Action) following the implementation of WGS in the borderer sense. Since it is not acceptable that patients throughout Europe are refused access to a cutting-edge diagnostic technology due to implementation problems. The participants who developed this Action propose to build a network that guides scientific developments enabling the successful implementation of new diagnostic concepts based on NGS technology. The initiative will unite leading consortia and stakeholders in one pan-European network.
List of Websites:
Website: www.3gb-TEST.eu
Contact: info@3gb-TEST.eu
In a two year EU funded (FP7) coordination support action (CSA) project: 3Gb-TEST, we explored the current status of whole genome sequencing to identify gaps and needs for its implementation in routine diagnostics. As history learned, novel genetic technologies, e.g. the many PCR based screening methods, often got implemented instantaneously, while proper validation only followed in performing the “diagnostic” tests. These blurring lines between research and diagnostics are usually caused by the limited number of samples in case of rare disorders. With all recent NGS (next generation sequencing) approaches a similar trend was observed, the myriad of equipment and available capture kits, targeted gene panels up to whole exome kits are being introduced instantly as diagnostic tools all with limited in house validation. Sequencing the total human genome (WGS), or a 3Gb-Test as we coined it, is coming close (probably within 5 years) to be implemented as a tool in healthcare. Therefore, our effort is early enough to explore and identify what needs to be in place before implementation. Workshops and expert meetings were organized to address the following topics:
-Wet lab innovations
-Bioinformatics’ tools
-Indications for WGS and its clinical utility
-Training of professionals
-Ethics
-Health technology assessment.
The outcome of 3Gb-TEST project is an extensive technology roadmap document and a guideline in which gaps and/or needs are specified. Amongst others the following major gaps or needs were identified:
-Lack of knowledge to interpret the non-coding regions of the genome
-Clinical, diagnostic trustworthiness of prediction and assessment tools is needed
-Sharing of genotype and phenotype information into the routine diagnostic practice is not in yet place
-Training laboratory and clinical staff to cope WGS data is needed
-Capacity building within Europe
-Regular update of NGS guidelines towards WGS is needed
Furthermore, it is critical that provisional recommendations on ethics are made to ensure that regulatory and expert bodies are able to develop standards. Use of relevant past experience in other domains could be utilized in order to be able to formulate such recommendations in the interim data gathering period.
Concluding, the overall consensus for the short term is that WGS is not yet ready for diagnostic use, but soon will be. We suggest to first expand the current WES analysis with CNV detection, by use of the novel WES capture kits with the capacity to also interpret genomic CNVs. This will replace Array analysis as a first line tool and is for the time being be the best (cost efficient) option. Further, if the price allows WGS could be used to read out the exome only and all genomic CNVs. This latter option would be an ideal way to bridge the gap between WES and full WGS, and add info to build up statistical (genotype) data on the non-coding regions of the genome.
Project Context and Objectives:
Introducing diagnostic applications of ‘3Gb-Testing’ in human genetics: What is still needed?
3Gb-TEST, a two year EU funded (FP7) coordination support action (CSA) project was set up to investigate the current status of whole genome sequencing to identify gaps and needs for its implementation in routine diagnostics. History has learned that novel genetic technologies, e.g. the many PCR based screening methods, often got implemented almost instantaneously, while validation followed in performing “diagnostic” tests. Blurring lines between research and diagnostics are usually caused by the limited number of samples in case of rare disorders. Similarly with all new NGS (next generation sequencing) approaches we noticed the same trends, equipment and many capture kits, targeted gene panels up to whole exome kits have been introduced almost instantly as diagnostic tests with limited in house validation.
Sequencing the total human genome (WGS), or a 3Gb-Test as we coined it, is close to be implemented as a tool in healthcare. Many ongoing research initiatives now evidence it’s putative strength. Sequence technologies are improving, getting faster and cheaper in generating large data sets. Bioinformatics’ tools perform increasingly better in detecting not only single nucleotide variations and insertion deletion variants (SNPs/INDELs), but also in detecting copy number variations and structural variants (CNVs/ SV’s). Although care has to be taken because none these “often in house” tools seem to be optimal yet. The same holds for variant effect predictor tools, variation between the different approaches can still be substantial, so that the only hard evidence can be found in wet lab ( in vitro or in vivo ) studies to proof the extend of the functional defects.
Since, fast and often sub-optimal implementation of novel technologies have been more rule than exception over the last few decades we hope to set the stage for WGS with this 3Gb-TEST project by defining what is missing and or needed while preparing a roadmap and guideline for its implementation.
With a total of eleven partners and embedded in a well-established network of collaborators the 3Gb-TEST project set out to address the following 7 objectives:
I. Exchange of experience and knowledge
II. Assessment of research resources, gaps, and other needs
III. Determine suitable applications for the new technology
IV. Health Technology Assessment (HTA):
V. The organization of an External Quality Assessment (EQA) scheme
VI. The organization of courses/symposia in the participant’s regions
VII. The writing of one or more guideline documents and a technology roadmap
Project Results:
With a total of eleven partners and embedded in a well-established network of collaborators the 3Gb-TEST project addressed the 7 objectives as follows:
I. Exchange of experience and knowledge: 3Gb-TEST has organized expert meetings two of which attached to the major international congress in our field (ESHG), exchange of views and insights helped to assess the current state of NGS and WGS in particular. Novel developments in the field were followed closely and showed us that research is moving fast but on the other hand that at many levels, equipment, bioinformatics software, and interpretation lots of variation still exists which urges to be cautious with the implementation of WGS.
Also the workshops and expert meetings permitted rich multi-disciplinary discussion around the complex ethical issues related to the clinical use of new sequencing technologies. Guidelines and experience are starting to accumulate, and it is envisaged that over the next few years further experience will allow guidelines to be honed and improved to better serve patients. For the moment some caution was urged in view of the difficulties of interpretation of results, and it was generally (but not unanimously) felt that with the present state of knowledge, opportunistic screening was not something that should be offered to all who undergo WGS testing
II. Assessment of research resources, gaps, and other needs: Wide range of Bioinformatics research are tools available, we catalogued around 180 in-silico bioinformatics tools. Overall outcomes for variant sets did not differ between different algorithms, but concordance is not complete and algorithms cannot be trusted as a single source of evidence. The involvement of phenotype information could improve the outcome of clinical-grade algorithms. Also a clear need for clinical grade variant databases to complement algorithms with available knowledge, also at phenotype level is detected.
III. Determine suitable applications for the new technology: Trends that have been identified over the last 5 years: NGS has been successfully implemented in the mainstream diagnostics, this after settling on European guidelines for NGS (Matthijs et al. 2015). Several successful approaches for different purposes can be mentioned, NIPT for prenatal numerical chromosomal aberrations, whole exome sequencing (WES) or panel-based NGS solutions for mutation detection in genetically heterogeneous conditions, depending on the required balance between high diagnostic yield and high analytical sensitivity, ampli-seq-based sequencing or multiple inversion probe (MIP)-based sequencing for ultra-deep sequencing applications, e.g. in molecular tumor diagnostics, or detection of somatic mosaicism. Soon whole genome sequence analysis (3Gb-testing) will be an effective and financially viable alternative to all targeted gene analysis approaches and could have an enormous impact on diagnostic centers replacing many existing molecular and cytogenetic tests. However, although the genome will provide all available information SNPs/INDELs and CNVs/SVs, the majority of the additional information from full genome sequence will be difficult to interpret. Because the normal variation is not known yet. So for the short term using novel WES capture kits with the capacity to also interpret exomic CNVs, in necessary followed by an array, might be the best option. Alternatively if the price allows WGS could be used to read out the exome only and all genomic CNVs. This latter option would be an ideal bridge between the WES an WGS and add info build up statistical data on the non-coding regions of the genome.
IV. Health Technology Assessment (HTA): A total of 16 systematic reviews were identified and summarised. A total of 55 model-based economic evaluations were identified. In general, resource use considerations were consistent. Most economic evaluations looked at resources involved with the test, staff, hospitalisation, and other medical procedures. Sources for test costs included re-imbursement rates, expert elicitation, laboratory charges and quotes from the technology’s manufacturer. No studies had precisely estimated costs using methods such as micro-costing. Economic evaluation should form an integral part of the HTA process because decisions to approve genetic tests for use in any EU member state has implications for finite healthcare budgets.
V. The organization of an External Quality Assessment (EQA) scheme: With the widespread adoption of Next Generation Sequencing (NGS) in diagnostic testing, the challenges for EQA scheme providers are that there is still little consensus on how to do diagnostic testing with an NGS based approach. Some labs have opted to replace Sanger sequencing based tests on small panels of genes, whilst others are implementing clinically relevant exome (WES) or whole genome (WGS) testing. In addition, many labs are still transitioning from using the technology in research to clinical diagnostics. The technology itself is also developing at a rapid rate with 2nd generation technologies already in widespread use, and the development of 3rd generation approaches nearing completion. With such variation, the challenge for EQA providers is to develop an EQA scheme that is generic, platform-agnostic and testing context independent whilst at the same time fitting seamlessly in to a laboratories workflow. Given the challenges outlined above, development of 3Gb-EQA schemes has been slow and currently there are only two EQA providers offering this type of activity; the College of American Pathologists (CAP) and EMQN. Both schemes have evolved independently and are method based EQAs applicable to labs using gene panels, WES or WGS but differ in the scope of testing required. Also the data received from the bio-informaticians varied because VCF file format is not a real standard format. So the first NGS EQA studies were varying in results and difficult to interpret.
VI. The organization of courses/symposia in the participant’s regions: 3Gb-TEST has organized 13 public symposia, courses, and workshops to exchange knowledge and ideas. These meetings were organized in eight different countries, the Netherlands, Germany, Italy, Greece, France, Sweden, the Czech Republic and the UK. In addition, members of the Consortium presented at other meetings, such as the 13th International Symposium on Mutation in the Genome in April 2015, which was co-sponsored by 3Gb-TEST. There were several scientific publications in peer-reviewed journals.
VII. The writing of one or more guideline documents and a technology roadmap:
Written information to the community, such as guideline document; Introducing diagnostic applications of ‘3Gb-Testing’ in human genetics: What is still needed? and a technology Roadmap have been produced. All data inclusive all other documents are available on the project web site: www.3Gb-TEST.eu. Due to the special design of the web site and the brochure, there was a lot of (enthusiastic) feedback from the scientific community.
Potential Impact:
With this 3Gb-TEST project we provided a forecast on the NGS/WGS developments. The future of molecular testing will pose a challenge to both laboratory and clinical geneticists, due to the wealth and complexity of the sequence information obtained. Clinical utility and anticipated difficulties of whole genome sequencing are discussed, as it is important that patients and families receive correct advice and appropriate management. It is also important to preclude mistakes with respect to ethics, quality issues, and over- or misinterpretation of data. Therefore, apart from technical issues and quality issues also ELSI (ethical, Leagal and Scocietal impact) issues were discussed with equal priortity.
Amongst others the following major gaps and needs were identified: 1. Lack of knowledge to interpret the non-coding regions of the genome; 2. Clinical, diagnostic trustworthiness of prediction and assessment tools which are an issue rather than the availability and range of options; 3. Too many factors are still in the way for introducing the sharing of genotype and phenotype information into the routine diagnostic practice; 4. Training laboratory and clinical staff to cope WGS data; 5. Capacity building within Europe; 6. Update of NGS guidelines towards WGS; 7. It is critical that provisional recommendations are made to ensure that regulatory and expert bodies are able to set standards and so the suggestion was made that relevant past experience in other domains could be utilized in order to be able to formulate such recommendations in the interim data gathering period.
The overall consensus for the short term is to first use the novel WES capture kits with the capacity to also interpret genomic CNVs, if necessary followed by an array analysis. As this might for the time being be the best (cost efficient) option. Further, if the price allows WGS could be used to read out the exome only and all genomic CNVs. This latter option would be an ideal way to bridge the gap between WES an WGS and add info to build up statistical (genotype) data on the non-coding regions of the genome. Meanwhile updating the various WGS guidelines is essential.
As, current gaps in our knowledge have been identified and research will to be initiated to bridge these gaps. The 3Gb-TEST project broad stakeholders together and ensure they are informed with respect to the desirable and undesirable developments. Furthermore an new initiative has been under taken to follow up this project with a Cost Action project (Target-Action) following the implementation of WGS in the borderer sense. Since it is not acceptable that patients throughout Europe are refused access to a cutting-edge diagnostic technology due to implementation problems. The participants who developed this Action propose to build a network that guides scientific developments enabling the successful implementation of new diagnostic concepts based on NGS technology. The initiative will unite leading consortia and stakeholders in one pan-European network.
List of Websites:
Website: www.3gb-TEST.eu
Contact: info@3gb-TEST.eu
