Final Report Summary - ISSC (An integrated surveillance system for infectious disease in rural China: generating evidence for early detection of disease epidemics in resource-poor settings.)
Executive Summary:
Syndromic surveillance system has great advantages in promoting the early detection of epidemics and reducing the necessities of disease confirmation. However, there are very few reports on the development of an electronic syndromic surveillance system in resource-constrained settings. Under the EU FP7 ISSC project, we developed and implemented a web-based syndromic surveillance system, called ISS system, in rural China and evaluated if it can improve the early detection of infectious disease epidemics under current conditions. The feasibility and acceptability of the system were also evaluated, as well as economic analysis.
The ISS system was installed in a Web hotel in Shanghai, China, in July 2011 and local users have operated it at study sites since Aug. 1, 2011. Since that day, a huge amount of data has been collected and transferred to the ISS system daily, including three different syndrome data, patients’ chief complaints, OTC medication sales, and school absenteeism. The first six months (Aug.1 2011-Jan.31 2012) was conducted as pilot study, during which the surveillance and analysis scheme were tested and modified. After that, the system has been implemented for two years (Apr.1 2012-Mar.31 2014) in four rural counties and data collected during this period was used for analysis and evaluation.
The results found the time and spatial trends of syndromic data from the ISS system were mostly different from the trend of the corresponding case data from the conventional case report system except a few exceptions. Among different syndromic data, the time trend of medication sales volume was found similar to that of patient symptoms/syndromes, especially for the respiratory symptoms/syndromes, and the time trends of cough, fever, ILI and ARI were found also similar to that of school absenteeism. When compared with real-time outbreak detection, the ISS system manifested a sensitivity of 80%-100% and predicative positive value (PPV) of 10%-60%, and the warning signals were 0-4 days generated before the epidemic peaks in the observed outbreaks. By comparing the signals between the ISS and case report system, the sensitivity and timeliness of syndromic surveillance were superior to those of case report system, but cases report system showed a higher PPV value. In simulated outbreak-based evaluation, school absenteeism surveillance manifested the best validity and timeliness, and drug sales surveillance showed the worst. In conclusion, syndromic surveillance system could supplement traditional case report system in the early warning of infectious disease outbreaks, and both surveillance systems played an important and unique role in the capture of infectious disease outbreak signals. The further integration and combination of the two systems could augment the overall outbreak detection capabilities in rural China.
The study found that local governments showed high political will and support for the implementation of the ISS system and different surveillance units managed to do the surveillance. The implementation was not without problems but as a whole it is technically feasible. This study also found that the ISS system which requires manual data reporting may generate more than small costs. But with the growing availability of electronic data sources, the system can be scalable to achieve automated data fetching with electronic data, which will make the system more sustainable and timely responsive.
Project Context and Objectives:
Introduction:
A crucial goal of infectious disease surveillance is the early detection of epidemics. In China, the current surveillance system is based on confirmed case reports and only covers notifiable diseases. However in rural China, it is not practical for health units to perform laboratory tests to confirm disease and people are more likely to get 'old' and emerging infectious diseases due to poor living conditions and closer contacts with wild animals and poultry. Thus, a sensitive and convenient early warning surveillance system for infectious disease is urgently needed in rural China. In 2010, European Commission started to fund a FP7 project on developing a syndromic surveillance system in rural China to complement the existing case reporting system (ISSC: 241900). The project is coordinated by the Division of Global Health, Karolinska Institutet, Stockholm. During this project, a new web-based syndromic surveillance system has been developed and implement in four rural counties, to promote the early detection of epidemics under current conditions.
Project setting:
The project involves three phases: (1) development of an integrated surveillance system-ISS; (2) implementation of ISS; (3) evaluation of ISS.
(1)Development of ISS
In ISS, the syndromic surveillance system was integrated with the case report surveillance system to enhance the capability of early detection of disease outbreaks. It involved the data collection for syndromic surveillance, the establishment of information systems and the development of automatic outbreak detection models.
(2) Implementation of ISS
The implementation includes two stages. Before the implementation, all participating personnel in experimental counties were first trained. The first six months (Aug.1 2011-Jan.31 2012) was conducted as pilot study, during which the surveillance and analysis scheme were tested and modified. After that, the system has been implemented for two years (Apr.1 2012-Mar.31 2014) in four rural counties. The daily surveillance data was analyzed. The alert signals triggered from syndromic surveillance were compared with those from case report surveillance.
(3) Evaluation of ISS
Through evaluation, three questions are answered: 1) is the capability of early warning improved through integrating syndromic surveillance into the conventional case report system? The early warning capability of the new added syndromic surveillance was compared against the conventional case report surveillance (as a reference group) using a field experimental study design.2) is ISS feasible and acceptable under current conditions in rural China? 3) is ISS cost-effective? The latter two evaluations were conducted in a before and after study design.
Study areas:
Four counties in two provinces (Hubei and Jiangxi) were selected as study areas. Hubei and Jiangxi provinces are both middle and low income regions in central China. Compared with the low income regions in western China, the two provinces are more densely populated (western areas are mostly sparsely populated in China) and thus more vulnerable to infectious diseases epidemics.
The population characteristics, economic development, and the occurrence patterns of infectious diseases in both provinces are similar. The per capita GDP of Hubei province was 1691USD, and Jiangxi was 1614USD in 2007. Hubei Province has a population density of 326 per square kilometer, and Jiangxi is 262, both of which are much higher than the average population density of China (137/sq.km). The proportion of people in rural areas in Hubei is 55.7%, Jiangxi 60.2%, which are close to the average level in China (55.06%). The life expectancy is 73.7 years in Hubei, 71.4 in Jiangxi, and average life expectancy is also 71.4 years in China at the time of project commencement. The incidence rate of notifiable infectious disease in Hubei is 310.33/100,000 Jiangxi 300.95/100,000 which are higher than the average level in China 272.39/100,000. The health authorities in the two provinces have expressed a strong willingness to take part in this project.
Because epidemics of infectious diseases are closely related to population density, in each province, counties are stratified into two levels, according to their population density. In each level, a county was purposively sampled considering the actual operational conditions for ISS. Thus, there are two sampled counties (one is much more densely populated, and the other is with relatively sparse population) in each province.
Diseases under study:
As respiratory infectious diseases, such as influenza, epidemic cerebrospinal meningitis, measles, chicken pox, mumps, SARS, etc. and gastrointestinal infectious diseases, such as bacillary dysentery, enteritis, viral hepatitis, polio, are the most common diseases in China, ISS mainly focused on the early detection of these two types of disease epidemics.
Syndromic data in ISS:
In the ISSC project, there are three different data sources of syndrome information, (1) patients’ chief complaints when they visit doctors; (2) OTC medication sales; (3) school absenteeism. These three data sources reflect different health related events before diagnosis, during the course of the development of infectious diseases. They have also been used frequently in syndromic surveillance and have provided the basis for some meaningful findings.
Aim
This project aims to develop an integrated surveillance system (ISS) for the early detection of infectious disease epidemics in rural China, which will integrate syndromic surveillance, conventional case report surveillance and Geographic Information System (GIS). In order to evaluate the early warning value of the new added syndromic surveillance, surveillance data series from the syndromic surveillance will be compared against those from the corresponding case report surveillance to assess the validity and timeliness of aberrant signal detection. Further, the acceptability, feasibility analysis and economic evaluation of the whole integrated surveillance system will be conducted and scientific suggestions will be put forward for decision-makers.
Specific Objectives
1 To develop an integrated surveillance system
An integrated surveillance system will combine syndromic surveillance, case report surveillance and GIS. The integrated system will be established based on the following four parts: 1) data collection and transfers; 2) data processing; 3) statistical description and analysis to detect ‘alert signals’ in temporal trends and linked to GIS to detect ‘alert signals’ in spatial trends; 4) further epidemiologic analysis, interpretation, investigation and laboratory test if necessary.
2 To conduct integrated surveillance in experimental counties
Four counties in two provinces will be selected as experimental sites. Daily data will be collected and analyzed to detect early aberration signals for epidemic. Factors crucial to the successful implementation of the integration will be identified.
3 To evaluate the integrated system
Include: 1) the assessment of the early warning capability of the new added syndromic surveillance. Surveillance data series produced by the syndromic surveillance will be compared with those from the conventional case report surveillance. The timeliness and validity of the ‘alert signals’ triggered from the syndromic surveillance will be evaluated in relation to the conventional case report surveillance system (as a reference); 2) the feasibility and acceptability analysis of the whole ISS will be conducted. In the interventional counties, the relevant personnel will be interviewed before and after the implementation of ISS to assess the acceptability and feasibility of the ISS; 3) economic evaluation of the ISS will also be analyzed in a before and after study.
Project Results:
Tables and figures for this part are included in the attachment.
Part One: Development of a web-based syndromic surveillance system
With reference to the ideas and experiences in the development of Real-time Outbreak and Disease Surveillance (RODS) and Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE II), a new syndromic surveillance system was developed (isscproject.com) by FPX, based on an existing platform ‘Crisis Information Sharing Platform’ (CRISP), combining modern communication with GIS technology.
1.Developing procedure
The work of developing the ISS system is based on FPX’s platform “CRISP” (Crisis Information Sharing Platform), which has been modified to satisfy the needs of ISS. The advantage of modifying an existing platform to adapt to ISS, is to secure the delivery of a large and intelligent system for ISSC project. The basic programming code has been already tested for other applications and a huge part of fundamental programming has been already tested and also verified.
a)Release of version 1.0 1st March 2011
The system was ready to use for inputting data in forms and begin a testing period. A lot of communications regarding how to develop a requested system for ISSC have been performed since the system was launched.
b)Workshop of ISS version 1.0 in Shanghai on 7th -8th March 2011
This workshop was used to give instructions on how to register into ISS, data collection interfaces in surveillance units and data import/ export functions.
c)Release of version 1.1 1st July 2011
On the date of July 1st, the system was installed on a web hotel in China. The system was ready to be tested and the speed of the system running went much faster for China users. The ISS system was hereby available in both English and Chinese languages.
d)Release of version 1.2 1st August 2011
On August 1st, system started to retrieve real live data reported from surveillance units in pilot study areas in China. Since that date, more than 20 000 records have been input into the database. System was now running as a domestic China system.
e)Release of version 1.3 10th January 2012
Some new functions were added in ISS system, such as the function of Dashboard, Alarm triggers and alarm models.
f)Release of version 1.4 1st August 2012
Data source in ISS system was updated, including search filter, language update, bug fixed, and export data source.
g)Release of version 1.5 14th September 2012
Dashboard and alarm trigger were updated, and several bugs in algorithms were fixed.
h)Release of version 1.6 1st November 2012
Digital polygons to handle alarm areas were implemented in system. The web map server was exchanged to a Chinese provider Baidu.
2.Structure of the ISS system
ISS has four interconnected components: 1) work group and communication group; 2) data source and collection; 3) data visualization; and 4) outbreak detection and alerting. Within each component, ISS offers users a variety of tools and options so as to create a customized and scalable electronic disease surveillance system according to users’ needs and capabilities.
Work groups and Communication groups
The work group and communication group module enables the system administrators to restrain data accessibilities to increase data security and privacy protection, yet by the same time facilitate the sharing of valuable information among authorized users.
Similar to other syndromic surveillance systems, ISS has a password-protected, encrypted Web site in which users can input, review, query, export, analyze and visualize surveillance data. When a user logs in, the system will check the user’s profile and only allow the user’s activities within his/her predefined authorities. System administrators have the right to create structural work groups, invite new users to different work groups and define authorities of different group members.
In the ISS platform, system administrators also can create communication groups and define group members for a specific communication group. Users in the same communication group can share information specifically belongs to that group. For example, they can set several alert triggers, and receive notifications of alerts generated from predefined alert triggers by email. The composition of a communication group is flexible. It may include one or more work groups or a subset of group members not necessarily a whole work group.
Data Sources and Collection
ISS collects daily syndrome information from three different data sources, including main symptoms of patients who present at health facilities, medication sales from retail pharmacies and primary school absenteeism. Data records are standardized and can be reported by a data collector either directly through filling of web forms by computer, or importing into the system via Excel files.
In ISS, system administrators are able to set different checking rules for data input. Once the data is reported and transferred into the central database, the system checks for logical erroneous and missing values, such as an onset date of symptoms is later than the patient’s visit date, the age value fall outside of reliable limits, body temperature is missing when the symptom of ‘fever’ is chosen, etc. Once the system detects any errors or missing, it will display a popup message on the web page, indicating that the procession of data transfer is failed and the reason for the failure. After having corrected the errors, data collectors submit the web form again to transfer data.
Data visualization
With the ISS platform, users can create a dashboard , which works as a desktop interface where users can add different widgets to present data in customized alarm lists, maps and charts. Dashboard can be related to a specific workgroup (workgroup dashboard) or a single user (personal dashboard). A ‘workgroup dashboard’ contains data that is linked to a specific work group while ‘personal dashboard’ contains data from all the workgroups that the user belongs to.
Alarm list present all alerts information, which belongs to one or several communication groups chronologically. This function allows users to check the detailed information of each alert in an aggregate way. Maps are created to present generated alerts geographically. The Chart screen provides a general plotting capability, in which the characteristics of data can be displayed in the format of lines, bars and pies.
Outbreak detection and alerting
The Alarm Trigger module incorporates models to detect aberrations of syndromic observations from daily surveillance data. When an alert is triggered, the system will automatically send e-mail notifications to subscribed communication groups. At this stage, the ISS has incorporated nine detection algorithms into the system, including models for temporal analysis, spatial analysis, and spatial-temporal analysis. ISS platform enables flexible and customizable model settings. The system sets default values for model parameters and threshold, but it also allows users to modify the model settings based upon needs. They can choose to run models for a specific data source or time interval. They can also set specific time for the system to run models automatically, such as to run a model every 8 hours per day or at 6:00 every morning, etc. Each time a detection model is run, a log will be generated and saved in the system, including data sources, model parameters, and detection time when the model threshold is exceeded to generate alerts. In order to get an overview of the running results and to evaluate different models, a comparison matrix is created when selecting more than one model to run using the same data source.
Since Aug.1st 2011, the system has run smoothly at surveillance sites, and a large amount of surveillance data has been input and analyzed in the system daily. The functions of the system have been improved continuously during the implementation period of the system. The system was uninstalled from the Chinese server on Sep. 9th, 2014.
Part Two: Main results of pilot study
The aims of pilot study include qualifying the trained participants, determining baselines and thresholds for ‘signal’ detection models and modifying the surveillance scheme and analytical model according to the actual surveillance results.
Methods
1. Sampling strategy
In each study county, one county hospital, one pharmacy and one primary school have been sampled for surveillance in pilot study at county level. At township level, at least one township has been sampled. Once the townships being fixed, township hospitals, township primary schools, township pharmacies, village health stations and village primary schools within the town/township administration range have simultaneously been included for surveillance.
2. Training strategy during pilot study
Before the initiation of ISS pilot study, the participants, including data collectors, field investigators, students and researches should receive tailored intensive training in ethics, basic knowledge on infectious diseases and surveillance, ISS (web-based system) functions and manual, SOPs for implementation, skills of investigation and data analysis for qualification. The on-site practice of ISS enables the participants to perform properly and effectively (details see WP3 work plan).
3. Data collection and analysis
Data collection and analysis have been conducted following the requirements of D1.1. The general distribution of targeted symptoms, syndroms and diseases; sales of drugs on the surveillance list; and student absenteeism in time, person and place have been described by surveillance sites at different levels
Control bars of Shewart, Moving Average (MA), Exponentially Weighted Moving Average (EWMA) and Cumulative Sum (CUSUM-C1, C2, and C3) were chosen to explore proper thresholds and parameter for alert models. Small Area Regression and Testing (SMART), Space-time Scan and What Is Strange about Recent Event (WSARE) have also been applied.
Main findings
1 General description of surveillance sites
According to the sampling strategy, 6 townships, 2 from Hubei (HB) and 4 from Jiangxi (JX), were sampled as the study sites in pilot study. Overall, 4 county hospitals, 6 township hospitals, 81 village health stations, 14 pharmacies and 24 primary schools were selected as surveillance units in the pilot study (Table 1).
2 Training for the pilot study
Before and during the pilot study, the participants, including data collectors, field investigators, students and researchers received tailored intensive training (Table 2).All the data collectors and supervisors were well trained for both theoretical and operational knowledge and technique of ISSC, and received certificates for the qualification of participating in the training. SOPs for ISSC project including data collection, transfer, entry, and modification were distributed to each data collector in pilot study.
3 Data reporting and quality control
Data collection network including healthcare facility, pharmacy and primary school was established, and daily data had been collected from health centers, pharmacies and primary schools. The reporting time was from Aug.1 2011 to Jan. 31, 2102 in pilot study with a report rate of 98.7%. Of all participating units, the proportion of “Missing report” , “Zero report” and “Event report” were 1.3% (237/17856), 12.4% (2216/17856), and 86.3% (15403/17856) respectively. Categorized by surveillance sources, the reports of targeted cases from county hospitals, township health centers and village clinics were 8171, 7207 and 92339 respectively. Total records of drug sales from pharmacy were 35665, and the absenteeism reports from primary schools were 2237.
Quality control forms had been developed and used to supervise the performance of data collectors. After qualifying the participants through training and on-site guidance, several quality control measures were applied to ensure the timeliness, accuracy, and completeness of data.
4 Data analysis
The general distribution in time, person and place of targeted symptoms, syndroms and diseases, sales of drugs on the surveillance list and student absenteeism were described by surveillance sites at different levels. Several models for thresholds setting and signal alerting were tested including Control bars of Shewart, Moving Average (MA), Exponentially Weighted Moving Average (EWMA), Cumulative Sum (CUSUM-C1, C2, C3), Small Area Regression and Testing (SMART), Space-time Scan and What Is Strange About Recent Event (WSARE). Based on the results of pilot data, modification for surveillance and analysis scheme were suggested.
5 Modification of the surveillance scheme and analysis model
According to the result of pilot study, the surveillance scheme and analysis models have been modified. Main modifications were presented below.
5.1 Sampling strategy
The sample size in formal study should cover 50% of total population in the study sites, i.e. approximate 150,000 persons in each study county. The suggested number of units was 4 towns in Hubei sites and 13 towns in Jiangxi sites.
5.2 Inclusion criteria of surveillance units
The selection of surveillance units in formal study was generally in accordance to the original criteria, but following aspects should be taken into consideration: computer and network configuration, capacity of data reporters and their willingness to participate in ISSC. In addition, apart from the requirement for the size of schools, in the formal implementation, schools with less than 5 grades, without internet access, or lack of data reporter would be excluded.
5.3 Surveillance contents
Two new items should be added to the surveillance contents of health care unit: preliminary diagnosis and initial/return visit. The other information including the ten targeted symptoms remained unchanged in the formal implementation of ISS. Drug sales volume could be influenced by consumer behaviors like stockpiling. Therefore, whenever possible the daily number of buyers of each drug should be added in the surveillance contents of pharmacies. One new item was added to the surveillance contents in primary school: initial/continuous absence, to differentiate students who were absent for multiple days from new absences.
5.4 Zero report and reporting on holidays
Zero reporting should be done as the original scheme except for days when the surveillance units were closed. On national holidays, a short delay in reporting should be allowed until the data collectors back to work.
5.5 Users authorization in ISSC system
In order to take the responsibility of supervising data reporting at village clinics, data collectors in township hospital should be authorized access to data sources at all the village clinics under the township, in terms of browsing, editing and adding new data. Moreover, under emergency conditions like the breakdown of internet or computer in a village, township data collectors with this authority should be able to report data on behalf of the village doctor through phone calls.
Part 3: Formal implementation of the ISS system in rural China
The aims of formal implementation include conducting analysis for surveillance data, comparing the ‘alert signals’ from the syndromic surveillance with those from the conventional case report surveillance system, and identifying factors that will be crucial for the successful implementation of such integrated system.
Methods
1 study subjects and surveillance contents
1.1 Data reported during the formal implementation period (April 1, 2012 to March 31, 2014) were extracted from the web-based ISS system for analysis, and the case data of notifiable diseases were extracted from the conventional case report system (CISDCP) as a reference for comparing.
Ten symptoms from health facilities, 98 kinds of OTC medications from pharmacies and school absenteeism from primary schools were targeted for surveillance in Hubei (Qianjiang and Shayang County) and Jiangxi (Fengxin and Yongxiu County) during the formal implementation (Table 3).
1.2 Participants involved in ISS were surveyed using a structural questionnaire to investigate the crucial factors influencing the successful implementation of ISS, including data collectors from health facilities, pharmacies and schools, as well as administrators and supervisors from local CDCs and health bureaus.
2. Data analysis
To identify the consistencies and differences between the specific syndromes from ISS and cases from CISDCP, the notifiable diseases were classified into three different categories: respiratory diseases, gastrointestinal diseases and rash-related diseases. One notifiable disease can be assigned to two or more different categories according to its main symptoms. Considering the actual situation of the reported data, only varicella and mumps in Hubei, and rash-related diseases in Jiangxi were selected to be compared with school absenteeism.
To compare the syndromic signals and case signals, the non-historical model of the Early Aberration Reporting System (EARS) was applied in Hubei (threshold of C3 set as 3), while the short-historical model of the Shewart model was employed in Jiangxi (parameters set as Baseline = 7 days, Lag=2, Sigma=3). For both models, the case signals were set as references. Sensitivity and predicted possible value (PPV) was calculated to indicate the validity of the models. Sensitivity was defined as the number of case signals that correctly corresponded to syndromic signals, divided by the total number of case signals. PPV was defined as the number of syndromic signals that corresponded to case signals, divided by the total number of syndromic signals. The results from Hubei by using EARS model and from Jiangxi by using Shewart model were respectively presented in Results 3.1 and 3.2.
Main Results:
1 Overview of data
Table 4 presents the overview of data collection during the formal implementation. The average outpatient volume was about 14, and less than five patients with targeted symptoms reported per surveillance unit each day. The daily average absenteeism per school was less than one.
2 Comparing the surveillance data from ISS and CISDCP
2.1 Comparisons of data from ISS and from CISDCP: Only a few comparisons showed similar time trends, 1) In Hubei: nausea/vomiting, diarrhea vs. gastrointestinal diseases in Qianjiang, antidiarrheals vs. gastrointestinal diseases in Shayang, school absenteeism vs. HFMD, chickenpox in both Qianjiang and Shayang. 2) In Jiangxi: nausea/vomiting, diarrhea vs. gastrointestinal diseases in Fengxin, fever-gastrointestinal syndrome vs. gastrointestinal diseases in Yongxiu, antidiarrheals vs. gastrointestinal diseases in Fengxin.
2.2 Comparisons of the three different data sources of ISS: the time trend of medication sales volume was found similar to that of symptoms/syndromes, especially for the respiratory symptoms/syndromes (the time trends of fever, cough, sore throat, ARI, and ILI all showed good consistency to that of compound cold medicine, cough suppressants, and antibiotics). In addition, the time trends of cough, fever, ILI and ARI were also similar to that of school absenteeism.
2.3 Comparisons of the spatial trends of data from ISS and from CISDCP: the spatial trends of data from ISS and from CISDCP were inconsistent.
3 Comparisons between signals triggered from both data sources
3.1 In Hubei: 1) Chief-complaint symptoms/syndromes and infectious diseases: cough, nausea/vomiting, and rash were the optimal symptoms/syndromes to detect aberrations of respiratory, gastrointestinal and rash-related diseases, respectively. Fever is the optimal symptom to detect aberrations of total infectious diseases, but fever+headache had higher sensitivity in Shayang. 2) Medication sales volume and infectious diseases: generally, antipyretics were the optimal medication to detect aberrations of infectious diseases; however, antibiotics achieved a higher PPV in detecting aberrations of total infectious diseases in Qianjiang, and antidiarrheals had a higher sensitivity in detecting aberrations of gastrointestinal diseases in Shayang. 3) School absenteeism and infectious diseases: in detecting the aberrations of varicella and mumps, absence on illness achieved a PPV of more than 70%, and the sensitivity was about 50% (Table 5).
3.2 In Jiangxi: 1) Chief-complaint symptoms/syndromes and infectious diseases: ILI, diarrhea, and rash were the optimal symptoms/syndromes to detect aberrations of respiratory, gastrointestinal and rash-related diseases, respectively. Fever in Fengxin and fever+headache in Yongxiu were the optimal symptom to detect aberrations of total infectious diseases 2) Medication sales volume and infectious diseases: generally, antipyretics and antibiotics were the optimal medication to detect aberrations of infectious diseases; however, in Yongxiu, compound cold medicine and antidiarrheals respectively achieved a higher sensitivity in detecting aberrations of respiratory diseases and gastrointestinal diseases. 3) School absenteeism and infectious diseases: in detecting the aberrations of rash-related diseases, school absenteeism achieved a higher sensitivity and PPV in Yongxiu(48.6%, 73.5%) than that in Fengxin(28.9%, 57.7%) (Table 6).
3.3 In summary, fever or fever with headache was the optimal symptoms to detect aberrations of total infectious diseases. Cough or ILI, diarrhea or nausea/vomiting, and rash respectively were the optimal symptoms/syndromes to detect aberrations of respiratory, gastrointestinal, and rash-related diseases. Antibiotics or antipyretics were the optimal medications to detect aberrations of infectious diseases. However, the sensitivity and PPV of most of the comparisons did not exceed 70%, which indicated that the ability of syndromic data to detect case signals is insufficient when data is from a general large scale; comparisons between signals triggered from ISS and from CISDCP should be more selective by restricting to a more specific location, population and diseases.
4 Crucial influencing factors for a successful implementation of ISS
268 and 289 eligible questionnaires were respectively collected from Hubei and Jiangxi sites with a response rate of 97.4% and 99.3% respectively. 57.4% participants in Hubei and 57.1% in Jiangxi Province expressed their willingness to participate in the similar work, however, the proportion declined to 28.7% (in Hubei) if there should be no payment. In addiction, very few participants (0.3% in Hubei and 1.4% in Jiangxi) feel very or fully satisfied with the payment for doing this project. Training and supervisions were also important factors. In Hubei, 56.6% participants thought daily supervision was important, 50.7% view that field supervision was very useful, 54.1% participants held the view that training was helpful for surveillance work (In Jiangxi, these three figures were 77.2%, 73.0%, 60.2%, respectively). Support from local bureau and government, and corporations with local CDCs should also be noted, because it would be very difficult to implement such a project in rural China without the help from these organizations.
Part Four: Early warning capability evaluation
The objective of this part include evaluating the timeliness and validity of syndrome ‘signals’ in outbreak detection.
1 Study design
To achieve the objectives, three approaches were applied for evaluation, including (1) evaluation based on real-time outbreak detection; (2) comparisons between the integrated syndromic surveillance (ISS) and case report system; and (3) evaluation based on simulated outbreak events. Since surveillance methods, strategies, and warning models in the field working were not completely same in Hubei and Jiangxi sites, the above approaches were performed in the two provinces respectively.
2 Data analysis
Surveillance units, whose reporting rate was less than 80% over the whole surveillance period, were excluded. When pharmacy promotion happened, the average sale volume of 1 day before and after that date was used instead of the actual drug sale volume. School absentees on business/injury/unknown were excluded and only the data of initial absence on illness was used for analysis. If their home address is not located in the surveillance sites, the cases either in ISS or case report system were excluded. The shewhart and EARS~3C models were used to generate warning signals in three parts, and the infectious diseases dynamic model and “high-fidelity” model were used to generate simulated outbreaks in the third parts.
Main results:
1.Hubei sites
1.1 evaluation based on real-time outbreak detection
There were four outbreaks in primary schools detected by ISS successfully, including two outbreaks in Shayang Experimental School (a varicella outbreak and a mumps outbreak), and an ILI outbreak in Shayang Shizhuang School, and one influenza B outbreak in Longwan Primary School. We estimated the timeliness and validity of ISS for outbreak detection through the above four outbreak events. Although timeliness of signals in different outbreak events was various, the warning signals were generated 0-2 days before the outbreak peaks in the four observed outbreaks.
The EARS~3C model was used to detect warning signals in Hubei site. The results showed a fine sensitivity for outbreak detection. The PPV values of warning model were not satisfactory, but the primary spatial assessment (checking whether the cases in signals occurred in the same schools or classrooms manually) is useful to reduce negative signals and improve the warning efficiency (the PPV value was higher in a combined warning model and primary spatial assessment than a single warning model). Overall, the ISS manifested 100% of sensitivities and 12.5%-60.0% of PPVs in the outbreak events (Table 7).
1.2Comparison evaluation
By comparing the signals between ISS and case report system, sensitivity and timeliness in syndromic surveillances were superior to those in case report system, but cases report system showed a higher PPV value. During the formal implementation period (from Apr. 1, 2012 to March 31, 2013), there were 29 signals occurred in the data stream of chief complaints, 29 in drug sales, 26 in school absenteeism, and 8 in case report system. The number of positive signals in the data stream of chief complaints, drug sales, absentees on illness, and case report system were 8, 8, 10, and 3 respectively. Therefore the sensitivity/PPV of each data stream could be calculated as 75.0%/27.6% for chief complaints surveillance, 100.0%/27.6% for drug sales surveillance, 100.0%/38.5% for school absenteeism surveillance, and 75.0%/37.5% for case report system.
In addition, the temporal trends of chief complaints, drug sales, and school absenteeism were found ahead of 7, 5, and 0 days of the case report system, respectively (Table 8).
1.3 Evaluation based on simulated outbreaks
In simulated outbreak-based evaluation, school absenteeism surveillance manifested the best validity and timeliness (the ROC curves were on the highest position, and AMOC curves were on the lowest position), and drug sales surveillance showed the worst validity and timeliness (the ROC curves of drug sales were on the lowest position, and AMOC curves were on the lowest position). (Figure 1)
2. Jiangxi Sites
2.1 evaluation based on real-time outbreak detection
During the formal implementation period, there were four outbreaks detected by ISS successfully, including an influenza A/H3N2 outbreak in Zaoxia Town in Fengxin County, and an influenza A/H1N1 in Zhuling Village, an influenza B in Mazhou Village, and a varicella outbreak in Yongxiu Couty. While there were two public health events not being detected by ISS, including an occupational poisoning event in Fengxin County and a paederus dermatitis outbreak in Yongxiu County. Among the nine true signals detected by ISSC, three were from Fengxin and six from Yongxiu. Eight out of nine true signals were proven to be influenza cluster or outbreak event while the other one was varicella outbreak. Most infectious disease events were detected among primary school surveillance sites, whereas no event from pharmacy surveillance sites. ISS manifested an 83.3% of sensitivity and 10% of PPV. The ISSC system triggered alert signals for two infectious disease events successfully and timely: the timeliness for detecting the new H1N1 influenza cluster in village is 4 day, and for detecting the seasonal H3N2 influenza outbreak in kindergarten is 1 day (time delay of the first signals between ISS and case report system). However the timeliness of other events could not be evaluated, because no signal was generated in case report system.
2.2 comparison evaluation
By comparing two surveillance systems, syndromic data sources including ILI and drug sales volume had early warning potentials towards particular infectious disease epidemics.
The most valuable findings of this part of study were the strong correlation between ILI counts from ISSC and Weekly Virus Positive rates (VPR) from national influenza surveillance system, and most of the correlation coefficients were greater than 0.5. ILI subgroups from ISSC system presented 0~5 weeks’ early warning timeliness (the time lag at the maximum correlation coefficient between two time-series data), and many groups such as the ILI(initial visit), ILI(3-4y), ILI(5-10y), ILI(11-17y) groups, had very good lead time towards VPR data. The sale volumes of cold medicine, antibiotics, antipyretics and cough suppressant also presented good timeliness towards influenza epidemics (time lags were 3 weeks, 5 weeks, 3 weeks, and 5 weeks respectively, but the correlation coefficients were much smaller than that in ILI data.
2.3 Evaluation based on simulated outbreaks
High-fidelity simulation revealed that syndromic surveillance system was capable to detect disease outbreaks at an early stage. The timeliness of detecting vacicella outbreak remained the highest whilst the timeliness of detecting influenza and foodborne outbreaks using “any symptom” group was the lowest. (Figure 2)
Conclusively, the ISSC could detect real outbreaks at an early stage successfully, however the practical timeliness might differ across different syndrome groups, disease epidemic periods and areas. School absenteeism surveillance is the most useful and suitable for outbreak detection in rural areas. It manifested the best timeliness, but drug sales surveillance might be the worst one among three data sources. ISS syndromic surveillance could improve the sensitivity of outbreak detection, but it should be used together with the case report surveillance.
Part Five: feasibility and acceptability evaluation
Aims
To evaluate the feasibility and acceptability of the ISSC for the early warning of epidemics in the four counties
Research questions:
1) What is the acceptability and feasibility to the CISDCP before the implementation of the ISSC?
2) What is the acceptability of different stakeholders to the ISSC both before the implementation and during the implementation of the ISSC?
3) What is the feasibility to implement the ISSC for the early warning of epidemics in terms of techniques and finance?
Study design
Acceptability can vary as time passes by due to concrete situation, such as the threat level posed by infectious disease outbreaks, perceived value of early detection, support for the methods of surveillance and resource fluctuation (Buehler et al. 2004). So does feasibility. Therefore, this study adopted a before-after study design, as shown in Figure 3: we learned the acceptability and feasibility before the operation of the ISSC and after 15 months of its implementation. We mainly adopted qualitative study methods, with a nested quantitative study before the operation of the ISSC. (Figure 3)
Data collection
We collected both quantitative and qualitative data before the implementation of the ISSC. For quantitative data, we adopted cluster random sampling strategy and investigated staff including health care professional in village clinics, township and county hospitals and county CDC, staff at township and county pharmacies, and health teachers in primary schools at village, township and county levels. Fifty percent of towns in each county were selected as study sites. All villages belonged to these corresponding towns were automatically included. All related institutes in sampling sites were selected. For the qualitative data, we conducted 63 in-depth interviews and 37 FGDs, and detailed subjects and sample sizes are shown in table 9. We collected another round of data with an in-depth interview method during the operation of the ISSC and interview subjects and sample sizes are presented in Table 10.
Main results
1 A summary on the evaluation of the case report system(CISDCP)
Staff from county CDCs got the highest score on knowledge on the control of infectious diseases compared with those from county hospitals, township hospitals and village clinics. If comparing scores among the counties, it was YX that had the highest.
Staff from county CDCs, county and township hospitals and village clinics showed high satisfaction towards the CISDCP (67% respondents were completely satisfied or very satisfied with only 1 (0.2%) very dissatisfied).
Directors of county CDCs perceived that their administration for the control of infectious diseases towards village, township and county-level health facilities faced different issues. Compared with the beginning years when implemented the CISDCP, most doctors from the county, township and village level health facilities said that data quality were improved and missing records were reduced. Doctors from county hospitals had too big volume of out-patients visits which sometimes led to miss reporting of infectious diseases. Doctors from township hospitals didn’t consider their outpatients visit as a burden. They, however, complained that they didn’t have enough laboratory equipment to confirm their clinical diagnosis. Village doctors tended not to report suspected cases. Public health staff at county and township level hospitals claimed that their work on the control of infectious diseases was easy compared with their other tasks. Staff at CDCs pointed out that they could only do limited microbiological detection and physical and chemical test, and that most of their samples needed to be delivered to upper level laboratories. Some doctors at township level said that if outbreak detection only depended on the automatic warning capacity of the CISDCP, there would be a relative long diagnosis delay as only cases confirmed by laboratory were reported to the CISDCP. CDCs and hospitals claimed limited human resource and financial support for the control of infectious diseases. Most staff we interviewed, including those from county CDCs, county and township level hospitals and village clinics perceived that the policy on infectious diseases was a double-edged sword for the control of outbreaks.
2 A summary on the evaluation of acceptability
Our quantitative data collected before the implementation of the ISSC identified three groups among village doctors: 1) “Positive cluster” (n=106), with high acceptability towards the ISSC; 2) “Negative cluster” (n=180), with neutral, although still not strongly negative, perceptions regarding to the ISSC; and 3) “Silent cluster” (n=114), with the degree of acceptability between the former two. Those who didn’t get any training on infectious disease control for the CISDCP were more often the members of the “Negative cluster” compared to those who got trained. People who were less educated were more often members of “Silent cluster” compared with those with higher education levels.
Before the implementation of the ISSC, according to our qualitative study, local governments as well as Bureaus of Health claimed high acceptability towards the ISSC. County Bureaus of Education expressed their support for primary schools in participating in the ISSC. The acceptability towards the ISSC among staff members at surveillance units, including county and township level hospitals, village clinics, primary schools and pharmacies, however, was not as high as that of local governments and their sectors.
The range of missing report rates grouped by types of surveillance units were from 0% to 9.98%, with 0.08%, 1.29% and 3.27% as first to third quartile. As a whole, health facilities in all four counties, including county and township level hospitals and village clinics, had the lowest rates of missing reports, followed by pharmacies and then primary schools. The range of timely report rates grouped by types of surveillance units were from 45.18% to 98.46%, with 75.82%, 81.40% and 89.51% as first to third quartile. Those data, to some extent, reflected the acceptability towards the ISSC.
3 A summary on the evaluation of feasibility
Before the implementation of the ISSC, local governments supported the implementation of the surveillance. Local CDCs had confidence in the implementation of the surveillance within the health system but with concerns. The government sectors claimed feasibility in doing the surveillance at primary schools although they admitted that it would face some difficulties. Coordination between government sectors could facilitate and had already facilitated the planning of the implementation at pharmacies but it was still perceived by people from the health system the most difficult part.
Directors of county hospitals claimed feasibility in doing the syndromic surveillance system (SSS). Outpatient doctors, as required by directors of their hospitals, claimed that they would collect data through their outpatient logs but they had several complaints. Potential data entry clerks showed their uncertain about future work load for the ISSC. Directors at township hospitals showed more concern than these at county level besides that they claimed possibility in doing the SSS. Staff members from the departments of public health at township hospitals claimed that they had already been overburdened by their primary responsibility and they had some complaints on their current working computers. Outpatient doctors showed a mix attitude towards the ISSC: some claimed the difficulty in doing it; some said the future tailored outpatient logs may even make their work easier. Knowledge and training on the importance and technical aspects was important for the implementation of the ISSC at village clinics. Some village clinics would encounter problems with computers and internet access and doctors claimed that they should receive subsidy for doing the ISSC. Computers and internet access would not be a concern in doing the surveillance except some schools at village level where either computer or internet were not available. The difficulty in doing the surveillance at primary schools, almost claimed by all, would be that the surveillance needs data every day, including zero report. Owners of pharmacies admitted that they did not want to participate in the surveillance. But as FDAs asked them to, then they would join in the surveillance. Some pharmacies at township level showed difficulty in reporting data via computers. Two out of four CDCs claimed shortage of staff members at CDCs for the investigations of signals generated by the ISSC. All directors of CDCs mentioned the shortage of capable staff members in performing field and epidemiological investigations.
During its implementation in the first 15 months, two FDAs kept involved in the surveillance but not the other two. CDCs in the four counties arranged staff members to the surveillance with different positions and staff time which showed different interactions among CDCs and surveillance units. The four county hospitals had computers and staff members capable to report data for the ISSC. Management staff from those four hospitals claimed that they required outpatient doctors to write records for outpatients with incentives. We still received a mix of feedback from outpatient doctors: some said the ISSC outpatient logs facilitated their writing whereas others complained about added work. Some managers of county and township hospitals claimed that data quality on outpatient visits may not be good enough. The ISSC, claimed both by staff from CDCs and management staff members at township hospitals, even though affected by the precise of outpatient logs, had helped improve the completeness of outpatient records due to intensive supervision. Village doctors we interviewed in general claimed that their reporting task for the ISSC did not take much time and they had already got used to the work. They seldom came across problems with the ISSC platform in reporting. Occasionally, their internet was slow which then may take them more time in reporting data. Staff members at township hospitals who supervised village doctors in general, however, showed concerns on the completeness of reported data by village doctors. In general, staff at pharmacies claimed that daily reporting only took them several minutes and they had got used to the work. They claimed that ISSC had almost no influence to their work. According to our interviews in counties FX and YX, teachers could easily manage to report data to the ISSC online platform. They mentioned that the difficult part was to consist on doing it every day. Four counties had varied numbers of signals from the ISSC for early warning of epidemics. YX and FX had more than those in Hubei province. Investigations consumed lots of staff time, especially at county CDCs. County CDC managed to investigate those signals, believed the prevention effect of their activities following signals but also showed concerns on sustainability if only depending on their current staff members in signal verification.
Part Six: Economic analysis
Aims
1) To identify, measure and value the cost of the syndromic surveillance system (SSS);
2) To identify, measure and value the effectiveness of the SSS;
3) To carry out an incremental cost-effectiveness analysis.
Frame and design of the study
1 Perspective of the cost analysis
We conducted this study from the perspective of managers of the surveillance to determine the costs of additional surveillance function.
2 Time horizon
This study covered the 17 months for system development (originally one year plus five months’ delay), the whole pilot period (8 months) and 15 months of the formal implementation period, which makes the operation of the SSS 23 months.
3 Type of analysis
The study is an incremental cost effectiveness analysis. The results are presented as incremental cost effectiveness ratios, with health effects expressed in reported records (e.g. of outpatient visits, student absenteeism and sales of medicines) to the SSS, and ICERs were expressed as incremental cost per reported record from, respectively, village clinics, township hospitals, county hospitals, primary schools and pharmacies.
4 Scope of this study
The activities, of which the costs are included in this study, are illustrated in Figure 4. Costs studied here are those incurred during the process of data collection, data processing, and data analysis and signal verification.
5 Data sources and data collection
This study was based on both secondary and primary data. Data source was presented in Table11. This economic evaluation study in general followed the sampling of ISSC. At the county level, we performed a census; we investigated all surveillance units at county level with our organizational questionnaires and carried out surveys with the staff questionnaires for all people involved in the SSS at those units. At the township level, we also adopted stratified cluster sampling as the ISSC project. Towns to be included in our sampling had to fulfil the following two requirements: 1) included in our baseline investigation before the implementation of the SSS, since we had sampled 50% towns of the four counties and purposely included those towns, which probably would later implement the ISSC project; 2) were finally intervention towns of the ISSC project. All surveillance units in included towns and their affiliated villages represented our study institutions. We investigated each unit with an organizational questionnaire, surveyed one data reporter from each primary school, pharmacy and village clinic with a staff questionnaire, and interviewed all staff involved in the SSS in township hospitals with staff questionnaires. We collected 369 organizational questionnaires and 477 staff questionnaires. Data reported to the ISSC Platform included 15 months’ data from 224 village clinics, 17 township hospitals, 4 county hospitals, primary schools from 43 villages, 18 at township and 13 at county level, as well as 18 and 26 pharmacies from, respectively, township and county level.
6 Data analysis
We classified costs into three groups: i) costs of system development; ii) of system operation and iii) of upgrading, with costs of training an independent part beyond the three. We developed a structure of costs of the SSS, based on activities and grouped by inputs, as shown in Figure 5.
We used local currency, Chinese RMB, in cost and CEA analysis, as local currency is the basis for further exchanging among currencies if needed. We did not provide costs in international dollar through purchasing power parity (ppp) exchange rate, for different methodology can generate different PPP exchange rates, which finally arrive at different international dollars.
Overall, staff time costs were major components of costs of system development, operation, upgrading and training. Staff time costs were measured by multiplying the number of full time employment equivalents with annual salaries and benefits, as suggested in several previous studies
The costs of computers included their depreciation, opportunity and operation costs. For depreciation and opportunity costs, we annualized computer purchasing costs according to their usage life and depreciation rate.
For daily operation costs of the SSS, besides staff costs, depreciation and opportunity costs of computers, all other costs were calculated by allocating corresponding overheads to the SSS. The equivalent full-time staff was as an allocation basis for overhead costs.
We imported each variable on reported records from health facilities, primary schools and pharmacies into Stata (StataCorp LP, Texas, USA) for descriptive analysis (minimum / P25 / P50 / P75 / maximum), and grouped the results by level (village, township and county) as well as by county (QJ, SY, FX and YX): this procedure allowed to identify and compare distribution of records within and among groups. .
We divided the total annual costs of each group, which were village clinics, township hospitals, county hospitals, primary schools and pharmacies, by their corresponding numbers of annual reported records to obtain the costs per reported record for each group.
Main results
Surveillance units in Jiangxi province (County FX and County YX) detected 36 raw signals during the 15 months of implementation, which was 9 times of that in Hubei province (County QJ and County SY). Health facilities in the two provinces detected 19 raw signals which was equal to the number of raw signals detected by primary schools, and which was 9.5 times of the number by pharmacies.
Daily average reported numbers of records per surveillance unit ranged from the highest to the lowest were, respectively, pharmacies, county hospitals, township hospitals, village clinics and primary schools. Varies existed within each of the five categories of surveillance units and also between Jiangxi and Hubei provinces. The numbers of reported records and rates of reported records per total outpatient volumes in health facilities in most cases were much higher in Jiangxi than those in Hubei province.
Cost per reported record from the five surveillance categories ranged from the highest to the lowest were, respectively, primary schools, county hospitals, township hospitals, village clinics and pharmacies. Cost per reported record in Jiangxi province in each of the five categories was higher than its corresponding in Hubei province.
Annual operation cost per surveillance unit ordered from the highest to the lowest were, respectively, pharmacies, county hospitals, village clinics, township hospitals and primary schools in Jiangxi province. The only difference in the order of annual operation costs per unit of the five categories between Jiangxi and Hubei was that in Hubei, the annual operation cost of a township hospital was higher than that of a village clinic. Annual operation costs per surveillance unit of the five categories in Jiangxi, again, were all higher than their corresponding in Hubei province.
When took a closer look at factors influencing costs, then time is a major resource for both the development and operation of the SSS. Time costs for data reporting, as variable costs, had the following characteristics: 1) at the initial stage of data reporting, more time was consumed compared with the time needed after familiarization with reporting. 2) daily reporting times varied within and across groups, but the median time for reporting in different groups was around 20 to 30 minutes with no big differences across groups. Together with the differences in annual salaries and daily total working hours, however, the opportunity costs of time in reporting data was observed to be extremely different within and across groups. 3) Most village doctors and around half of staff members from primary schools claimed to report data after normal work hours while others mainly worked during normal working time.
Time costs in data collection and reporting, daily management of data reporting and overhead costs allocated to the SSS were the main components of operational costs. If grouped by the categories of surveillance units, it became evident that village clinics incurred most of total costs of the SSS in each county.
Organizational approaches for daily management of data reporting, field trips for data management, etc. varied between Hubei and Jiangxi province. Huge differences on costs existed between Hubei and Jiangxi province, including costs for system development, training, daily data management, field trips, etc. Costs for signal verification were negligible in this study. This study found that a SSS which requires manual data reporting may generate more than small costs.
Potential Impact:
Infectious disease is one of the major challenges to human progress and survival for centuries. In order to mitigate potential damages, early warning of infectious disease epidemics has long been one of the fundamental tasks for infectious diseases surveillance and a primary concern for public health institutions. In the ISSC project, we developed and implemented a web-based syndromic surveillance system, called ISS system in rural China. Using a field experimental study design, we evaluate if it can improve the early detection of infectious disease epidemics under current conditions. The feasibility and acceptability of the system were also evaluated, as well as economic analysis. The ISSC project will have an impact at several levels in relation to an early warning surveillance system for infectious diseases.
On the level of overall public health, the critical role of an early warning system for disease outbreak has been demonstrated in disease prevention and control practices. Timely detection of infectious disease outbreaks can lead to timely and effective response activities, so that morbidity, mortality and unnecessary suffering as well as transmission to other areas or countries are minimized.
This study showed syndromic surveillance system could supplement traditional case report system in the early warning of infectious disease outbreaks, and both systems played an important and unique role in the capture of infectious disease outbreak signals. By comparing the signals between the syndromic surveillance and case report system, sensitivity and timeliness of syndromic surveillance were superior to those of case report system, but cases report system showed a higher positive predictive value. The further integration and combination of the two systems could augment the overall outbreak detection capabilities in rural China.
ISSC is also an innovative project. There has been practically no information on this research area (An integrated early warning surveillance system in rural areas) from China (or Asia). Although syndromic surveillance as an innovative tool for early detection of disease outbreaks has attracted a great deal of attention, most research in this field has been conducted in developed areas or countries where modern information technology infrastructure is well established and professional human resource support is available. Until the current study, there are no public reports on the development and implementation of a syndromic surveillance system in rural China. The ISSC project has filled a gap in knowledge here.
On the level of the individual patient and infected person, an early warning surveillance system can help to identify, confirm and treat patients and infected persons in a timely way, to avoid more serious adverse effects on them and their families, since emergent response systems are based on early warning surveillance systems. This is of great value for people, especially for rural people living in resource-limited environments who are most heavily affected by epidemics.
On the level of the health system and policy, developing, implementing and evaluating an integrated surveillance system which aims at the early detection of disease outbreaks will provide valuable evidence, experience and knowledge to policy makers. The development of the early warning surveillance system for infectious diseases is a great concern of Chinese government and is highly relevant to national policy and to program makers in China, as well as in other countries. With the strong political support we see a window of opportunity to develop and implement a syndromic surveillance system in rural China. The project will also provide data to policy makers on pre-clinical and clinical pre-diagnostic information in rural China. The different data streams used in ISS can help to explore the contribution of multiple data sources to early detection and the monitoring of infectious disease outbreaks.
Getting research into policy and practice is a major goal of our project. The project results showed local governments and their sectors demonstrated high political will and support both before and during the implementation of the ISSC, and the implementation seems technically feasible as a whole. From research project to practice, the cooperation of local government, public health departments and CDCs are the key to its success. Project information has been disseminated to policy makers and experts regularly during the implementation of ISSC, and they commented ISSC provides a classical example to improve infectious disease surveillance in rural China, and gives important public health implications.
In addition, we also found several constraints and challenges from the study. Electronic data is lacking for some data sources and manual labor is needed for data collection, which decreased the compliance of data collectors and increasing the difficulties of data quality control. A multi-sector cooperation and data sharing strategy between different data streams owners are also lacking, which made it challenging to collect syndromic data from different sectors. Despite the challenges regarding data collection and sharing, the ISS system showed significant advantages over the conventional surveillance system. With the growing availability of electronic data sources and the establishment of data sharing policy, the system will be more sustainable and timely responsive.
The main dissemination activities and the exploitation of results:
The dissemination of the results has followed two tracks: academic and policy maker (government) dissemination. The academic dissemination has been achieved by way of a number of scientific conference presentations coming out of the ISSC project. A number of publications have also generated from this project, and some manuscripts of the findings have been prepared or are currently under review. A list of conferences at which ISSC project outcomes have been presented is below, as a list of publications.
There have also been a number of meetings with policy makers from Ministry of Health, China CDC, provincial & municipal health authorities, provincial, municipal, county and district CDCs, Bureau of Education, Bureau of Food and Drug Safety throughout the life of the project, but particularly towards the end of the project.
The ISSC final Dissemination Conference titled as “The Development and Application of Syndromic Surveillance System in Rural China” was held in Shanghai on October 21-22 2014. More than one hundred participants from six countries and regions took part in the conference, including representatives from partner institutions, officials from the Public Health Agency of Sweden, European CDC, China CDC and the project administration office of EC granted projects; experts from different provincial CDCs in China; scholars from several Chinese universities, and National Institute for Communicable Diseases, South Africa, and R.D. Gardi Medical College, Madhya Pradesh, India. Representatives from Shanghai Medical Association, Shanghai Preventive Medicine Association and other civil society have also joined the conferences. The conference presented a comprehensive demonstration of the ISSC by covering the project designs, development of network platform, capacity building, global dissemination, fieldwork in China (Hubei and Jiangxi) and its outputs/outcomes.
Nine briefs of ISSC have been made and disseminated to policy makers and researchers during the project implementation. They have also been disseminated at different international and national conferences.
Syndromic surveillance system has great advantages in promoting the early detection of epidemics and reducing the necessities of disease confirmation. However, there are very few reports on the development of an electronic syndromic surveillance system in resource-constrained settings. Under the EU FP7 ISSC project, we developed and implemented a web-based syndromic surveillance system, called ISS system, in rural China and evaluated if it can improve the early detection of infectious disease epidemics under current conditions. The feasibility and acceptability of the system were also evaluated, as well as economic analysis.
The ISS system was installed in a Web hotel in Shanghai, China, in July 2011 and local users have operated it at study sites since Aug. 1, 2011. Since that day, a huge amount of data has been collected and transferred to the ISS system daily, including three different syndrome data, patients’ chief complaints, OTC medication sales, and school absenteeism. The first six months (Aug.1 2011-Jan.31 2012) was conducted as pilot study, during which the surveillance and analysis scheme were tested and modified. After that, the system has been implemented for two years (Apr.1 2012-Mar.31 2014) in four rural counties and data collected during this period was used for analysis and evaluation.
The results found the time and spatial trends of syndromic data from the ISS system were mostly different from the trend of the corresponding case data from the conventional case report system except a few exceptions. Among different syndromic data, the time trend of medication sales volume was found similar to that of patient symptoms/syndromes, especially for the respiratory symptoms/syndromes, and the time trends of cough, fever, ILI and ARI were found also similar to that of school absenteeism. When compared with real-time outbreak detection, the ISS system manifested a sensitivity of 80%-100% and predicative positive value (PPV) of 10%-60%, and the warning signals were 0-4 days generated before the epidemic peaks in the observed outbreaks. By comparing the signals between the ISS and case report system, the sensitivity and timeliness of syndromic surveillance were superior to those of case report system, but cases report system showed a higher PPV value. In simulated outbreak-based evaluation, school absenteeism surveillance manifested the best validity and timeliness, and drug sales surveillance showed the worst. In conclusion, syndromic surveillance system could supplement traditional case report system in the early warning of infectious disease outbreaks, and both surveillance systems played an important and unique role in the capture of infectious disease outbreak signals. The further integration and combination of the two systems could augment the overall outbreak detection capabilities in rural China.
The study found that local governments showed high political will and support for the implementation of the ISS system and different surveillance units managed to do the surveillance. The implementation was not without problems but as a whole it is technically feasible. This study also found that the ISS system which requires manual data reporting may generate more than small costs. But with the growing availability of electronic data sources, the system can be scalable to achieve automated data fetching with electronic data, which will make the system more sustainable and timely responsive.
Project Context and Objectives:
Introduction:
A crucial goal of infectious disease surveillance is the early detection of epidemics. In China, the current surveillance system is based on confirmed case reports and only covers notifiable diseases. However in rural China, it is not practical for health units to perform laboratory tests to confirm disease and people are more likely to get 'old' and emerging infectious diseases due to poor living conditions and closer contacts with wild animals and poultry. Thus, a sensitive and convenient early warning surveillance system for infectious disease is urgently needed in rural China. In 2010, European Commission started to fund a FP7 project on developing a syndromic surveillance system in rural China to complement the existing case reporting system (ISSC: 241900). The project is coordinated by the Division of Global Health, Karolinska Institutet, Stockholm. During this project, a new web-based syndromic surveillance system has been developed and implement in four rural counties, to promote the early detection of epidemics under current conditions.
Project setting:
The project involves three phases: (1) development of an integrated surveillance system-ISS; (2) implementation of ISS; (3) evaluation of ISS.
(1)Development of ISS
In ISS, the syndromic surveillance system was integrated with the case report surveillance system to enhance the capability of early detection of disease outbreaks. It involved the data collection for syndromic surveillance, the establishment of information systems and the development of automatic outbreak detection models.
(2) Implementation of ISS
The implementation includes two stages. Before the implementation, all participating personnel in experimental counties were first trained. The first six months (Aug.1 2011-Jan.31 2012) was conducted as pilot study, during which the surveillance and analysis scheme were tested and modified. After that, the system has been implemented for two years (Apr.1 2012-Mar.31 2014) in four rural counties. The daily surveillance data was analyzed. The alert signals triggered from syndromic surveillance were compared with those from case report surveillance.
(3) Evaluation of ISS
Through evaluation, three questions are answered: 1) is the capability of early warning improved through integrating syndromic surveillance into the conventional case report system? The early warning capability of the new added syndromic surveillance was compared against the conventional case report surveillance (as a reference group) using a field experimental study design.2) is ISS feasible and acceptable under current conditions in rural China? 3) is ISS cost-effective? The latter two evaluations were conducted in a before and after study design.
Study areas:
Four counties in two provinces (Hubei and Jiangxi) were selected as study areas. Hubei and Jiangxi provinces are both middle and low income regions in central China. Compared with the low income regions in western China, the two provinces are more densely populated (western areas are mostly sparsely populated in China) and thus more vulnerable to infectious diseases epidemics.
The population characteristics, economic development, and the occurrence patterns of infectious diseases in both provinces are similar. The per capita GDP of Hubei province was 1691USD, and Jiangxi was 1614USD in 2007. Hubei Province has a population density of 326 per square kilometer, and Jiangxi is 262, both of which are much higher than the average population density of China (137/sq.km). The proportion of people in rural areas in Hubei is 55.7%, Jiangxi 60.2%, which are close to the average level in China (55.06%). The life expectancy is 73.7 years in Hubei, 71.4 in Jiangxi, and average life expectancy is also 71.4 years in China at the time of project commencement. The incidence rate of notifiable infectious disease in Hubei is 310.33/100,000 Jiangxi 300.95/100,000 which are higher than the average level in China 272.39/100,000. The health authorities in the two provinces have expressed a strong willingness to take part in this project.
Because epidemics of infectious diseases are closely related to population density, in each province, counties are stratified into two levels, according to their population density. In each level, a county was purposively sampled considering the actual operational conditions for ISS. Thus, there are two sampled counties (one is much more densely populated, and the other is with relatively sparse population) in each province.
Diseases under study:
As respiratory infectious diseases, such as influenza, epidemic cerebrospinal meningitis, measles, chicken pox, mumps, SARS, etc. and gastrointestinal infectious diseases, such as bacillary dysentery, enteritis, viral hepatitis, polio, are the most common diseases in China, ISS mainly focused on the early detection of these two types of disease epidemics.
Syndromic data in ISS:
In the ISSC project, there are three different data sources of syndrome information, (1) patients’ chief complaints when they visit doctors; (2) OTC medication sales; (3) school absenteeism. These three data sources reflect different health related events before diagnosis, during the course of the development of infectious diseases. They have also been used frequently in syndromic surveillance and have provided the basis for some meaningful findings.
Aim
This project aims to develop an integrated surveillance system (ISS) for the early detection of infectious disease epidemics in rural China, which will integrate syndromic surveillance, conventional case report surveillance and Geographic Information System (GIS). In order to evaluate the early warning value of the new added syndromic surveillance, surveillance data series from the syndromic surveillance will be compared against those from the corresponding case report surveillance to assess the validity and timeliness of aberrant signal detection. Further, the acceptability, feasibility analysis and economic evaluation of the whole integrated surveillance system will be conducted and scientific suggestions will be put forward for decision-makers.
Specific Objectives
1 To develop an integrated surveillance system
An integrated surveillance system will combine syndromic surveillance, case report surveillance and GIS. The integrated system will be established based on the following four parts: 1) data collection and transfers; 2) data processing; 3) statistical description and analysis to detect ‘alert signals’ in temporal trends and linked to GIS to detect ‘alert signals’ in spatial trends; 4) further epidemiologic analysis, interpretation, investigation and laboratory test if necessary.
2 To conduct integrated surveillance in experimental counties
Four counties in two provinces will be selected as experimental sites. Daily data will be collected and analyzed to detect early aberration signals for epidemic. Factors crucial to the successful implementation of the integration will be identified.
3 To evaluate the integrated system
Include: 1) the assessment of the early warning capability of the new added syndromic surveillance. Surveillance data series produced by the syndromic surveillance will be compared with those from the conventional case report surveillance. The timeliness and validity of the ‘alert signals’ triggered from the syndromic surveillance will be evaluated in relation to the conventional case report surveillance system (as a reference); 2) the feasibility and acceptability analysis of the whole ISS will be conducted. In the interventional counties, the relevant personnel will be interviewed before and after the implementation of ISS to assess the acceptability and feasibility of the ISS; 3) economic evaluation of the ISS will also be analyzed in a before and after study.
Project Results:
Tables and figures for this part are included in the attachment.
Part One: Development of a web-based syndromic surveillance system
With reference to the ideas and experiences in the development of Real-time Outbreak and Disease Surveillance (RODS) and Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE II), a new syndromic surveillance system was developed (isscproject.com) by FPX, based on an existing platform ‘Crisis Information Sharing Platform’ (CRISP), combining modern communication with GIS technology.
1.Developing procedure
The work of developing the ISS system is based on FPX’s platform “CRISP” (Crisis Information Sharing Platform), which has been modified to satisfy the needs of ISS. The advantage of modifying an existing platform to adapt to ISS, is to secure the delivery of a large and intelligent system for ISSC project. The basic programming code has been already tested for other applications and a huge part of fundamental programming has been already tested and also verified.
a)Release of version 1.0 1st March 2011
The system was ready to use for inputting data in forms and begin a testing period. A lot of communications regarding how to develop a requested system for ISSC have been performed since the system was launched.
b)Workshop of ISS version 1.0 in Shanghai on 7th -8th March 2011
This workshop was used to give instructions on how to register into ISS, data collection interfaces in surveillance units and data import/ export functions.
c)Release of version 1.1 1st July 2011
On the date of July 1st, the system was installed on a web hotel in China. The system was ready to be tested and the speed of the system running went much faster for China users. The ISS system was hereby available in both English and Chinese languages.
d)Release of version 1.2 1st August 2011
On August 1st, system started to retrieve real live data reported from surveillance units in pilot study areas in China. Since that date, more than 20 000 records have been input into the database. System was now running as a domestic China system.
e)Release of version 1.3 10th January 2012
Some new functions were added in ISS system, such as the function of Dashboard, Alarm triggers and alarm models.
f)Release of version 1.4 1st August 2012
Data source in ISS system was updated, including search filter, language update, bug fixed, and export data source.
g)Release of version 1.5 14th September 2012
Dashboard and alarm trigger were updated, and several bugs in algorithms were fixed.
h)Release of version 1.6 1st November 2012
Digital polygons to handle alarm areas were implemented in system. The web map server was exchanged to a Chinese provider Baidu.
2.Structure of the ISS system
ISS has four interconnected components: 1) work group and communication group; 2) data source and collection; 3) data visualization; and 4) outbreak detection and alerting. Within each component, ISS offers users a variety of tools and options so as to create a customized and scalable electronic disease surveillance system according to users’ needs and capabilities.
Work groups and Communication groups
The work group and communication group module enables the system administrators to restrain data accessibilities to increase data security and privacy protection, yet by the same time facilitate the sharing of valuable information among authorized users.
Similar to other syndromic surveillance systems, ISS has a password-protected, encrypted Web site in which users can input, review, query, export, analyze and visualize surveillance data. When a user logs in, the system will check the user’s profile and only allow the user’s activities within his/her predefined authorities. System administrators have the right to create structural work groups, invite new users to different work groups and define authorities of different group members.
In the ISS platform, system administrators also can create communication groups and define group members for a specific communication group. Users in the same communication group can share information specifically belongs to that group. For example, they can set several alert triggers, and receive notifications of alerts generated from predefined alert triggers by email. The composition of a communication group is flexible. It may include one or more work groups or a subset of group members not necessarily a whole work group.
Data Sources and Collection
ISS collects daily syndrome information from three different data sources, including main symptoms of patients who present at health facilities, medication sales from retail pharmacies and primary school absenteeism. Data records are standardized and can be reported by a data collector either directly through filling of web forms by computer, or importing into the system via Excel files.
In ISS, system administrators are able to set different checking rules for data input. Once the data is reported and transferred into the central database, the system checks for logical erroneous and missing values, such as an onset date of symptoms is later than the patient’s visit date, the age value fall outside of reliable limits, body temperature is missing when the symptom of ‘fever’ is chosen, etc. Once the system detects any errors or missing, it will display a popup message on the web page, indicating that the procession of data transfer is failed and the reason for the failure. After having corrected the errors, data collectors submit the web form again to transfer data.
Data visualization
With the ISS platform, users can create a dashboard , which works as a desktop interface where users can add different widgets to present data in customized alarm lists, maps and charts. Dashboard can be related to a specific workgroup (workgroup dashboard) or a single user (personal dashboard). A ‘workgroup dashboard’ contains data that is linked to a specific work group while ‘personal dashboard’ contains data from all the workgroups that the user belongs to.
Alarm list present all alerts information, which belongs to one or several communication groups chronologically. This function allows users to check the detailed information of each alert in an aggregate way. Maps are created to present generated alerts geographically. The Chart screen provides a general plotting capability, in which the characteristics of data can be displayed in the format of lines, bars and pies.
Outbreak detection and alerting
The Alarm Trigger module incorporates models to detect aberrations of syndromic observations from daily surveillance data. When an alert is triggered, the system will automatically send e-mail notifications to subscribed communication groups. At this stage, the ISS has incorporated nine detection algorithms into the system, including models for temporal analysis, spatial analysis, and spatial-temporal analysis. ISS platform enables flexible and customizable model settings. The system sets default values for model parameters and threshold, but it also allows users to modify the model settings based upon needs. They can choose to run models for a specific data source or time interval. They can also set specific time for the system to run models automatically, such as to run a model every 8 hours per day or at 6:00 every morning, etc. Each time a detection model is run, a log will be generated and saved in the system, including data sources, model parameters, and detection time when the model threshold is exceeded to generate alerts. In order to get an overview of the running results and to evaluate different models, a comparison matrix is created when selecting more than one model to run using the same data source.
Since Aug.1st 2011, the system has run smoothly at surveillance sites, and a large amount of surveillance data has been input and analyzed in the system daily. The functions of the system have been improved continuously during the implementation period of the system. The system was uninstalled from the Chinese server on Sep. 9th, 2014.
Part Two: Main results of pilot study
The aims of pilot study include qualifying the trained participants, determining baselines and thresholds for ‘signal’ detection models and modifying the surveillance scheme and analytical model according to the actual surveillance results.
Methods
1. Sampling strategy
In each study county, one county hospital, one pharmacy and one primary school have been sampled for surveillance in pilot study at county level. At township level, at least one township has been sampled. Once the townships being fixed, township hospitals, township primary schools, township pharmacies, village health stations and village primary schools within the town/township administration range have simultaneously been included for surveillance.
2. Training strategy during pilot study
Before the initiation of ISS pilot study, the participants, including data collectors, field investigators, students and researches should receive tailored intensive training in ethics, basic knowledge on infectious diseases and surveillance, ISS (web-based system) functions and manual, SOPs for implementation, skills of investigation and data analysis for qualification. The on-site practice of ISS enables the participants to perform properly and effectively (details see WP3 work plan).
3. Data collection and analysis
Data collection and analysis have been conducted following the requirements of D1.1. The general distribution of targeted symptoms, syndroms and diseases; sales of drugs on the surveillance list; and student absenteeism in time, person and place have been described by surveillance sites at different levels
Control bars of Shewart, Moving Average (MA), Exponentially Weighted Moving Average (EWMA) and Cumulative Sum (CUSUM-C1, C2, and C3) were chosen to explore proper thresholds and parameter for alert models. Small Area Regression and Testing (SMART), Space-time Scan and What Is Strange about Recent Event (WSARE) have also been applied.
Main findings
1 General description of surveillance sites
According to the sampling strategy, 6 townships, 2 from Hubei (HB) and 4 from Jiangxi (JX), were sampled as the study sites in pilot study. Overall, 4 county hospitals, 6 township hospitals, 81 village health stations, 14 pharmacies and 24 primary schools were selected as surveillance units in the pilot study (Table 1).
2 Training for the pilot study
Before and during the pilot study, the participants, including data collectors, field investigators, students and researchers received tailored intensive training (Table 2).All the data collectors and supervisors were well trained for both theoretical and operational knowledge and technique of ISSC, and received certificates for the qualification of participating in the training. SOPs for ISSC project including data collection, transfer, entry, and modification were distributed to each data collector in pilot study.
3 Data reporting and quality control
Data collection network including healthcare facility, pharmacy and primary school was established, and daily data had been collected from health centers, pharmacies and primary schools. The reporting time was from Aug.1 2011 to Jan. 31, 2102 in pilot study with a report rate of 98.7%. Of all participating units, the proportion of “Missing report” , “Zero report” and “Event report” were 1.3% (237/17856), 12.4% (2216/17856), and 86.3% (15403/17856) respectively. Categorized by surveillance sources, the reports of targeted cases from county hospitals, township health centers and village clinics were 8171, 7207 and 92339 respectively. Total records of drug sales from pharmacy were 35665, and the absenteeism reports from primary schools were 2237.
Quality control forms had been developed and used to supervise the performance of data collectors. After qualifying the participants through training and on-site guidance, several quality control measures were applied to ensure the timeliness, accuracy, and completeness of data.
4 Data analysis
The general distribution in time, person and place of targeted symptoms, syndroms and diseases, sales of drugs on the surveillance list and student absenteeism were described by surveillance sites at different levels. Several models for thresholds setting and signal alerting were tested including Control bars of Shewart, Moving Average (MA), Exponentially Weighted Moving Average (EWMA), Cumulative Sum (CUSUM-C1, C2, C3), Small Area Regression and Testing (SMART), Space-time Scan and What Is Strange About Recent Event (WSARE). Based on the results of pilot data, modification for surveillance and analysis scheme were suggested.
5 Modification of the surveillance scheme and analysis model
According to the result of pilot study, the surveillance scheme and analysis models have been modified. Main modifications were presented below.
5.1 Sampling strategy
The sample size in formal study should cover 50% of total population in the study sites, i.e. approximate 150,000 persons in each study county. The suggested number of units was 4 towns in Hubei sites and 13 towns in Jiangxi sites.
5.2 Inclusion criteria of surveillance units
The selection of surveillance units in formal study was generally in accordance to the original criteria, but following aspects should be taken into consideration: computer and network configuration, capacity of data reporters and their willingness to participate in ISSC. In addition, apart from the requirement for the size of schools, in the formal implementation, schools with less than 5 grades, without internet access, or lack of data reporter would be excluded.
5.3 Surveillance contents
Two new items should be added to the surveillance contents of health care unit: preliminary diagnosis and initial/return visit. The other information including the ten targeted symptoms remained unchanged in the formal implementation of ISS. Drug sales volume could be influenced by consumer behaviors like stockpiling. Therefore, whenever possible the daily number of buyers of each drug should be added in the surveillance contents of pharmacies. One new item was added to the surveillance contents in primary school: initial/continuous absence, to differentiate students who were absent for multiple days from new absences.
5.4 Zero report and reporting on holidays
Zero reporting should be done as the original scheme except for days when the surveillance units were closed. On national holidays, a short delay in reporting should be allowed until the data collectors back to work.
5.5 Users authorization in ISSC system
In order to take the responsibility of supervising data reporting at village clinics, data collectors in township hospital should be authorized access to data sources at all the village clinics under the township, in terms of browsing, editing and adding new data. Moreover, under emergency conditions like the breakdown of internet or computer in a village, township data collectors with this authority should be able to report data on behalf of the village doctor through phone calls.
Part 3: Formal implementation of the ISS system in rural China
The aims of formal implementation include conducting analysis for surveillance data, comparing the ‘alert signals’ from the syndromic surveillance with those from the conventional case report surveillance system, and identifying factors that will be crucial for the successful implementation of such integrated system.
Methods
1 study subjects and surveillance contents
1.1 Data reported during the formal implementation period (April 1, 2012 to March 31, 2014) were extracted from the web-based ISS system for analysis, and the case data of notifiable diseases were extracted from the conventional case report system (CISDCP) as a reference for comparing.
Ten symptoms from health facilities, 98 kinds of OTC medications from pharmacies and school absenteeism from primary schools were targeted for surveillance in Hubei (Qianjiang and Shayang County) and Jiangxi (Fengxin and Yongxiu County) during the formal implementation (Table 3).
1.2 Participants involved in ISS were surveyed using a structural questionnaire to investigate the crucial factors influencing the successful implementation of ISS, including data collectors from health facilities, pharmacies and schools, as well as administrators and supervisors from local CDCs and health bureaus.
2. Data analysis
To identify the consistencies and differences between the specific syndromes from ISS and cases from CISDCP, the notifiable diseases were classified into three different categories: respiratory diseases, gastrointestinal diseases and rash-related diseases. One notifiable disease can be assigned to two or more different categories according to its main symptoms. Considering the actual situation of the reported data, only varicella and mumps in Hubei, and rash-related diseases in Jiangxi were selected to be compared with school absenteeism.
To compare the syndromic signals and case signals, the non-historical model of the Early Aberration Reporting System (EARS) was applied in Hubei (threshold of C3 set as 3), while the short-historical model of the Shewart model was employed in Jiangxi (parameters set as Baseline = 7 days, Lag=2, Sigma=3). For both models, the case signals were set as references. Sensitivity and predicted possible value (PPV) was calculated to indicate the validity of the models. Sensitivity was defined as the number of case signals that correctly corresponded to syndromic signals, divided by the total number of case signals. PPV was defined as the number of syndromic signals that corresponded to case signals, divided by the total number of syndromic signals. The results from Hubei by using EARS model and from Jiangxi by using Shewart model were respectively presented in Results 3.1 and 3.2.
Main Results:
1 Overview of data
Table 4 presents the overview of data collection during the formal implementation. The average outpatient volume was about 14, and less than five patients with targeted symptoms reported per surveillance unit each day. The daily average absenteeism per school was less than one.
2 Comparing the surveillance data from ISS and CISDCP
2.1 Comparisons of data from ISS and from CISDCP: Only a few comparisons showed similar time trends, 1) In Hubei: nausea/vomiting, diarrhea vs. gastrointestinal diseases in Qianjiang, antidiarrheals vs. gastrointestinal diseases in Shayang, school absenteeism vs. HFMD, chickenpox in both Qianjiang and Shayang. 2) In Jiangxi: nausea/vomiting, diarrhea vs. gastrointestinal diseases in Fengxin, fever-gastrointestinal syndrome vs. gastrointestinal diseases in Yongxiu, antidiarrheals vs. gastrointestinal diseases in Fengxin.
2.2 Comparisons of the three different data sources of ISS: the time trend of medication sales volume was found similar to that of symptoms/syndromes, especially for the respiratory symptoms/syndromes (the time trends of fever, cough, sore throat, ARI, and ILI all showed good consistency to that of compound cold medicine, cough suppressants, and antibiotics). In addition, the time trends of cough, fever, ILI and ARI were also similar to that of school absenteeism.
2.3 Comparisons of the spatial trends of data from ISS and from CISDCP: the spatial trends of data from ISS and from CISDCP were inconsistent.
3 Comparisons between signals triggered from both data sources
3.1 In Hubei: 1) Chief-complaint symptoms/syndromes and infectious diseases: cough, nausea/vomiting, and rash were the optimal symptoms/syndromes to detect aberrations of respiratory, gastrointestinal and rash-related diseases, respectively. Fever is the optimal symptom to detect aberrations of total infectious diseases, but fever+headache had higher sensitivity in Shayang. 2) Medication sales volume and infectious diseases: generally, antipyretics were the optimal medication to detect aberrations of infectious diseases; however, antibiotics achieved a higher PPV in detecting aberrations of total infectious diseases in Qianjiang, and antidiarrheals had a higher sensitivity in detecting aberrations of gastrointestinal diseases in Shayang. 3) School absenteeism and infectious diseases: in detecting the aberrations of varicella and mumps, absence on illness achieved a PPV of more than 70%, and the sensitivity was about 50% (Table 5).
3.2 In Jiangxi: 1) Chief-complaint symptoms/syndromes and infectious diseases: ILI, diarrhea, and rash were the optimal symptoms/syndromes to detect aberrations of respiratory, gastrointestinal and rash-related diseases, respectively. Fever in Fengxin and fever+headache in Yongxiu were the optimal symptom to detect aberrations of total infectious diseases 2) Medication sales volume and infectious diseases: generally, antipyretics and antibiotics were the optimal medication to detect aberrations of infectious diseases; however, in Yongxiu, compound cold medicine and antidiarrheals respectively achieved a higher sensitivity in detecting aberrations of respiratory diseases and gastrointestinal diseases. 3) School absenteeism and infectious diseases: in detecting the aberrations of rash-related diseases, school absenteeism achieved a higher sensitivity and PPV in Yongxiu(48.6%, 73.5%) than that in Fengxin(28.9%, 57.7%) (Table 6).
3.3 In summary, fever or fever with headache was the optimal symptoms to detect aberrations of total infectious diseases. Cough or ILI, diarrhea or nausea/vomiting, and rash respectively were the optimal symptoms/syndromes to detect aberrations of respiratory, gastrointestinal, and rash-related diseases. Antibiotics or antipyretics were the optimal medications to detect aberrations of infectious diseases. However, the sensitivity and PPV of most of the comparisons did not exceed 70%, which indicated that the ability of syndromic data to detect case signals is insufficient when data is from a general large scale; comparisons between signals triggered from ISS and from CISDCP should be more selective by restricting to a more specific location, population and diseases.
4 Crucial influencing factors for a successful implementation of ISS
268 and 289 eligible questionnaires were respectively collected from Hubei and Jiangxi sites with a response rate of 97.4% and 99.3% respectively. 57.4% participants in Hubei and 57.1% in Jiangxi Province expressed their willingness to participate in the similar work, however, the proportion declined to 28.7% (in Hubei) if there should be no payment. In addiction, very few participants (0.3% in Hubei and 1.4% in Jiangxi) feel very or fully satisfied with the payment for doing this project. Training and supervisions were also important factors. In Hubei, 56.6% participants thought daily supervision was important, 50.7% view that field supervision was very useful, 54.1% participants held the view that training was helpful for surveillance work (In Jiangxi, these three figures were 77.2%, 73.0%, 60.2%, respectively). Support from local bureau and government, and corporations with local CDCs should also be noted, because it would be very difficult to implement such a project in rural China without the help from these organizations.
Part Four: Early warning capability evaluation
The objective of this part include evaluating the timeliness and validity of syndrome ‘signals’ in outbreak detection.
1 Study design
To achieve the objectives, three approaches were applied for evaluation, including (1) evaluation based on real-time outbreak detection; (2) comparisons between the integrated syndromic surveillance (ISS) and case report system; and (3) evaluation based on simulated outbreak events. Since surveillance methods, strategies, and warning models in the field working were not completely same in Hubei and Jiangxi sites, the above approaches were performed in the two provinces respectively.
2 Data analysis
Surveillance units, whose reporting rate was less than 80% over the whole surveillance period, were excluded. When pharmacy promotion happened, the average sale volume of 1 day before and after that date was used instead of the actual drug sale volume. School absentees on business/injury/unknown were excluded and only the data of initial absence on illness was used for analysis. If their home address is not located in the surveillance sites, the cases either in ISS or case report system were excluded. The shewhart and EARS~3C models were used to generate warning signals in three parts, and the infectious diseases dynamic model and “high-fidelity” model were used to generate simulated outbreaks in the third parts.
Main results:
1.Hubei sites
1.1 evaluation based on real-time outbreak detection
There were four outbreaks in primary schools detected by ISS successfully, including two outbreaks in Shayang Experimental School (a varicella outbreak and a mumps outbreak), and an ILI outbreak in Shayang Shizhuang School, and one influenza B outbreak in Longwan Primary School. We estimated the timeliness and validity of ISS for outbreak detection through the above four outbreak events. Although timeliness of signals in different outbreak events was various, the warning signals were generated 0-2 days before the outbreak peaks in the four observed outbreaks.
The EARS~3C model was used to detect warning signals in Hubei site. The results showed a fine sensitivity for outbreak detection. The PPV values of warning model were not satisfactory, but the primary spatial assessment (checking whether the cases in signals occurred in the same schools or classrooms manually) is useful to reduce negative signals and improve the warning efficiency (the PPV value was higher in a combined warning model and primary spatial assessment than a single warning model). Overall, the ISS manifested 100% of sensitivities and 12.5%-60.0% of PPVs in the outbreak events (Table 7).
1.2Comparison evaluation
By comparing the signals between ISS and case report system, sensitivity and timeliness in syndromic surveillances were superior to those in case report system, but cases report system showed a higher PPV value. During the formal implementation period (from Apr. 1, 2012 to March 31, 2013), there were 29 signals occurred in the data stream of chief complaints, 29 in drug sales, 26 in school absenteeism, and 8 in case report system. The number of positive signals in the data stream of chief complaints, drug sales, absentees on illness, and case report system were 8, 8, 10, and 3 respectively. Therefore the sensitivity/PPV of each data stream could be calculated as 75.0%/27.6% for chief complaints surveillance, 100.0%/27.6% for drug sales surveillance, 100.0%/38.5% for school absenteeism surveillance, and 75.0%/37.5% for case report system.
In addition, the temporal trends of chief complaints, drug sales, and school absenteeism were found ahead of 7, 5, and 0 days of the case report system, respectively (Table 8).
1.3 Evaluation based on simulated outbreaks
In simulated outbreak-based evaluation, school absenteeism surveillance manifested the best validity and timeliness (the ROC curves were on the highest position, and AMOC curves were on the lowest position), and drug sales surveillance showed the worst validity and timeliness (the ROC curves of drug sales were on the lowest position, and AMOC curves were on the lowest position). (Figure 1)
2. Jiangxi Sites
2.1 evaluation based on real-time outbreak detection
During the formal implementation period, there were four outbreaks detected by ISS successfully, including an influenza A/H3N2 outbreak in Zaoxia Town in Fengxin County, and an influenza A/H1N1 in Zhuling Village, an influenza B in Mazhou Village, and a varicella outbreak in Yongxiu Couty. While there were two public health events not being detected by ISS, including an occupational poisoning event in Fengxin County and a paederus dermatitis outbreak in Yongxiu County. Among the nine true signals detected by ISSC, three were from Fengxin and six from Yongxiu. Eight out of nine true signals were proven to be influenza cluster or outbreak event while the other one was varicella outbreak. Most infectious disease events were detected among primary school surveillance sites, whereas no event from pharmacy surveillance sites. ISS manifested an 83.3% of sensitivity and 10% of PPV. The ISSC system triggered alert signals for two infectious disease events successfully and timely: the timeliness for detecting the new H1N1 influenza cluster in village is 4 day, and for detecting the seasonal H3N2 influenza outbreak in kindergarten is 1 day (time delay of the first signals between ISS and case report system). However the timeliness of other events could not be evaluated, because no signal was generated in case report system.
2.2 comparison evaluation
By comparing two surveillance systems, syndromic data sources including ILI and drug sales volume had early warning potentials towards particular infectious disease epidemics.
The most valuable findings of this part of study were the strong correlation between ILI counts from ISSC and Weekly Virus Positive rates (VPR) from national influenza surveillance system, and most of the correlation coefficients were greater than 0.5. ILI subgroups from ISSC system presented 0~5 weeks’ early warning timeliness (the time lag at the maximum correlation coefficient between two time-series data), and many groups such as the ILI(initial visit), ILI(3-4y), ILI(5-10y), ILI(11-17y) groups, had very good lead time towards VPR data. The sale volumes of cold medicine, antibiotics, antipyretics and cough suppressant also presented good timeliness towards influenza epidemics (time lags were 3 weeks, 5 weeks, 3 weeks, and 5 weeks respectively, but the correlation coefficients were much smaller than that in ILI data.
2.3 Evaluation based on simulated outbreaks
High-fidelity simulation revealed that syndromic surveillance system was capable to detect disease outbreaks at an early stage. The timeliness of detecting vacicella outbreak remained the highest whilst the timeliness of detecting influenza and foodborne outbreaks using “any symptom” group was the lowest. (Figure 2)
Conclusively, the ISSC could detect real outbreaks at an early stage successfully, however the practical timeliness might differ across different syndrome groups, disease epidemic periods and areas. School absenteeism surveillance is the most useful and suitable for outbreak detection in rural areas. It manifested the best timeliness, but drug sales surveillance might be the worst one among three data sources. ISS syndromic surveillance could improve the sensitivity of outbreak detection, but it should be used together with the case report surveillance.
Part Five: feasibility and acceptability evaluation
Aims
To evaluate the feasibility and acceptability of the ISSC for the early warning of epidemics in the four counties
Research questions:
1) What is the acceptability and feasibility to the CISDCP before the implementation of the ISSC?
2) What is the acceptability of different stakeholders to the ISSC both before the implementation and during the implementation of the ISSC?
3) What is the feasibility to implement the ISSC for the early warning of epidemics in terms of techniques and finance?
Study design
Acceptability can vary as time passes by due to concrete situation, such as the threat level posed by infectious disease outbreaks, perceived value of early detection, support for the methods of surveillance and resource fluctuation (Buehler et al. 2004). So does feasibility. Therefore, this study adopted a before-after study design, as shown in Figure 3: we learned the acceptability and feasibility before the operation of the ISSC and after 15 months of its implementation. We mainly adopted qualitative study methods, with a nested quantitative study before the operation of the ISSC. (Figure 3)
Data collection
We collected both quantitative and qualitative data before the implementation of the ISSC. For quantitative data, we adopted cluster random sampling strategy and investigated staff including health care professional in village clinics, township and county hospitals and county CDC, staff at township and county pharmacies, and health teachers in primary schools at village, township and county levels. Fifty percent of towns in each county were selected as study sites. All villages belonged to these corresponding towns were automatically included. All related institutes in sampling sites were selected. For the qualitative data, we conducted 63 in-depth interviews and 37 FGDs, and detailed subjects and sample sizes are shown in table 9. We collected another round of data with an in-depth interview method during the operation of the ISSC and interview subjects and sample sizes are presented in Table 10.
Main results
1 A summary on the evaluation of the case report system(CISDCP)
Staff from county CDCs got the highest score on knowledge on the control of infectious diseases compared with those from county hospitals, township hospitals and village clinics. If comparing scores among the counties, it was YX that had the highest.
Staff from county CDCs, county and township hospitals and village clinics showed high satisfaction towards the CISDCP (67% respondents were completely satisfied or very satisfied with only 1 (0.2%) very dissatisfied).
Directors of county CDCs perceived that their administration for the control of infectious diseases towards village, township and county-level health facilities faced different issues. Compared with the beginning years when implemented the CISDCP, most doctors from the county, township and village level health facilities said that data quality were improved and missing records were reduced. Doctors from county hospitals had too big volume of out-patients visits which sometimes led to miss reporting of infectious diseases. Doctors from township hospitals didn’t consider their outpatients visit as a burden. They, however, complained that they didn’t have enough laboratory equipment to confirm their clinical diagnosis. Village doctors tended not to report suspected cases. Public health staff at county and township level hospitals claimed that their work on the control of infectious diseases was easy compared with their other tasks. Staff at CDCs pointed out that they could only do limited microbiological detection and physical and chemical test, and that most of their samples needed to be delivered to upper level laboratories. Some doctors at township level said that if outbreak detection only depended on the automatic warning capacity of the CISDCP, there would be a relative long diagnosis delay as only cases confirmed by laboratory were reported to the CISDCP. CDCs and hospitals claimed limited human resource and financial support for the control of infectious diseases. Most staff we interviewed, including those from county CDCs, county and township level hospitals and village clinics perceived that the policy on infectious diseases was a double-edged sword for the control of outbreaks.
2 A summary on the evaluation of acceptability
Our quantitative data collected before the implementation of the ISSC identified three groups among village doctors: 1) “Positive cluster” (n=106), with high acceptability towards the ISSC; 2) “Negative cluster” (n=180), with neutral, although still not strongly negative, perceptions regarding to the ISSC; and 3) “Silent cluster” (n=114), with the degree of acceptability between the former two. Those who didn’t get any training on infectious disease control for the CISDCP were more often the members of the “Negative cluster” compared to those who got trained. People who were less educated were more often members of “Silent cluster” compared with those with higher education levels.
Before the implementation of the ISSC, according to our qualitative study, local governments as well as Bureaus of Health claimed high acceptability towards the ISSC. County Bureaus of Education expressed their support for primary schools in participating in the ISSC. The acceptability towards the ISSC among staff members at surveillance units, including county and township level hospitals, village clinics, primary schools and pharmacies, however, was not as high as that of local governments and their sectors.
The range of missing report rates grouped by types of surveillance units were from 0% to 9.98%, with 0.08%, 1.29% and 3.27% as first to third quartile. As a whole, health facilities in all four counties, including county and township level hospitals and village clinics, had the lowest rates of missing reports, followed by pharmacies and then primary schools. The range of timely report rates grouped by types of surveillance units were from 45.18% to 98.46%, with 75.82%, 81.40% and 89.51% as first to third quartile. Those data, to some extent, reflected the acceptability towards the ISSC.
3 A summary on the evaluation of feasibility
Before the implementation of the ISSC, local governments supported the implementation of the surveillance. Local CDCs had confidence in the implementation of the surveillance within the health system but with concerns. The government sectors claimed feasibility in doing the surveillance at primary schools although they admitted that it would face some difficulties. Coordination between government sectors could facilitate and had already facilitated the planning of the implementation at pharmacies but it was still perceived by people from the health system the most difficult part.
Directors of county hospitals claimed feasibility in doing the syndromic surveillance system (SSS). Outpatient doctors, as required by directors of their hospitals, claimed that they would collect data through their outpatient logs but they had several complaints. Potential data entry clerks showed their uncertain about future work load for the ISSC. Directors at township hospitals showed more concern than these at county level besides that they claimed possibility in doing the SSS. Staff members from the departments of public health at township hospitals claimed that they had already been overburdened by their primary responsibility and they had some complaints on their current working computers. Outpatient doctors showed a mix attitude towards the ISSC: some claimed the difficulty in doing it; some said the future tailored outpatient logs may even make their work easier. Knowledge and training on the importance and technical aspects was important for the implementation of the ISSC at village clinics. Some village clinics would encounter problems with computers and internet access and doctors claimed that they should receive subsidy for doing the ISSC. Computers and internet access would not be a concern in doing the surveillance except some schools at village level where either computer or internet were not available. The difficulty in doing the surveillance at primary schools, almost claimed by all, would be that the surveillance needs data every day, including zero report. Owners of pharmacies admitted that they did not want to participate in the surveillance. But as FDAs asked them to, then they would join in the surveillance. Some pharmacies at township level showed difficulty in reporting data via computers. Two out of four CDCs claimed shortage of staff members at CDCs for the investigations of signals generated by the ISSC. All directors of CDCs mentioned the shortage of capable staff members in performing field and epidemiological investigations.
During its implementation in the first 15 months, two FDAs kept involved in the surveillance but not the other two. CDCs in the four counties arranged staff members to the surveillance with different positions and staff time which showed different interactions among CDCs and surveillance units. The four county hospitals had computers and staff members capable to report data for the ISSC. Management staff from those four hospitals claimed that they required outpatient doctors to write records for outpatients with incentives. We still received a mix of feedback from outpatient doctors: some said the ISSC outpatient logs facilitated their writing whereas others complained about added work. Some managers of county and township hospitals claimed that data quality on outpatient visits may not be good enough. The ISSC, claimed both by staff from CDCs and management staff members at township hospitals, even though affected by the precise of outpatient logs, had helped improve the completeness of outpatient records due to intensive supervision. Village doctors we interviewed in general claimed that their reporting task for the ISSC did not take much time and they had already got used to the work. They seldom came across problems with the ISSC platform in reporting. Occasionally, their internet was slow which then may take them more time in reporting data. Staff members at township hospitals who supervised village doctors in general, however, showed concerns on the completeness of reported data by village doctors. In general, staff at pharmacies claimed that daily reporting only took them several minutes and they had got used to the work. They claimed that ISSC had almost no influence to their work. According to our interviews in counties FX and YX, teachers could easily manage to report data to the ISSC online platform. They mentioned that the difficult part was to consist on doing it every day. Four counties had varied numbers of signals from the ISSC for early warning of epidemics. YX and FX had more than those in Hubei province. Investigations consumed lots of staff time, especially at county CDCs. County CDC managed to investigate those signals, believed the prevention effect of their activities following signals but also showed concerns on sustainability if only depending on their current staff members in signal verification.
Part Six: Economic analysis
Aims
1) To identify, measure and value the cost of the syndromic surveillance system (SSS);
2) To identify, measure and value the effectiveness of the SSS;
3) To carry out an incremental cost-effectiveness analysis.
Frame and design of the study
1 Perspective of the cost analysis
We conducted this study from the perspective of managers of the surveillance to determine the costs of additional surveillance function.
2 Time horizon
This study covered the 17 months for system development (originally one year plus five months’ delay), the whole pilot period (8 months) and 15 months of the formal implementation period, which makes the operation of the SSS 23 months.
3 Type of analysis
The study is an incremental cost effectiveness analysis. The results are presented as incremental cost effectiveness ratios, with health effects expressed in reported records (e.g. of outpatient visits, student absenteeism and sales of medicines) to the SSS, and ICERs were expressed as incremental cost per reported record from, respectively, village clinics, township hospitals, county hospitals, primary schools and pharmacies.
4 Scope of this study
The activities, of which the costs are included in this study, are illustrated in Figure 4. Costs studied here are those incurred during the process of data collection, data processing, and data analysis and signal verification.
5 Data sources and data collection
This study was based on both secondary and primary data. Data source was presented in Table11. This economic evaluation study in general followed the sampling of ISSC. At the county level, we performed a census; we investigated all surveillance units at county level with our organizational questionnaires and carried out surveys with the staff questionnaires for all people involved in the SSS at those units. At the township level, we also adopted stratified cluster sampling as the ISSC project. Towns to be included in our sampling had to fulfil the following two requirements: 1) included in our baseline investigation before the implementation of the SSS, since we had sampled 50% towns of the four counties and purposely included those towns, which probably would later implement the ISSC project; 2) were finally intervention towns of the ISSC project. All surveillance units in included towns and their affiliated villages represented our study institutions. We investigated each unit with an organizational questionnaire, surveyed one data reporter from each primary school, pharmacy and village clinic with a staff questionnaire, and interviewed all staff involved in the SSS in township hospitals with staff questionnaires. We collected 369 organizational questionnaires and 477 staff questionnaires. Data reported to the ISSC Platform included 15 months’ data from 224 village clinics, 17 township hospitals, 4 county hospitals, primary schools from 43 villages, 18 at township and 13 at county level, as well as 18 and 26 pharmacies from, respectively, township and county level.
6 Data analysis
We classified costs into three groups: i) costs of system development; ii) of system operation and iii) of upgrading, with costs of training an independent part beyond the three. We developed a structure of costs of the SSS, based on activities and grouped by inputs, as shown in Figure 5.
We used local currency, Chinese RMB, in cost and CEA analysis, as local currency is the basis for further exchanging among currencies if needed. We did not provide costs in international dollar through purchasing power parity (ppp) exchange rate, for different methodology can generate different PPP exchange rates, which finally arrive at different international dollars.
Overall, staff time costs were major components of costs of system development, operation, upgrading and training. Staff time costs were measured by multiplying the number of full time employment equivalents with annual salaries and benefits, as suggested in several previous studies
The costs of computers included their depreciation, opportunity and operation costs. For depreciation and opportunity costs, we annualized computer purchasing costs according to their usage life and depreciation rate.
For daily operation costs of the SSS, besides staff costs, depreciation and opportunity costs of computers, all other costs were calculated by allocating corresponding overheads to the SSS. The equivalent full-time staff was as an allocation basis for overhead costs.
We imported each variable on reported records from health facilities, primary schools and pharmacies into Stata (StataCorp LP, Texas, USA) for descriptive analysis (minimum / P25 / P50 / P75 / maximum), and grouped the results by level (village, township and county) as well as by county (QJ, SY, FX and YX): this procedure allowed to identify and compare distribution of records within and among groups. .
We divided the total annual costs of each group, which were village clinics, township hospitals, county hospitals, primary schools and pharmacies, by their corresponding numbers of annual reported records to obtain the costs per reported record for each group.
Main results
Surveillance units in Jiangxi province (County FX and County YX) detected 36 raw signals during the 15 months of implementation, which was 9 times of that in Hubei province (County QJ and County SY). Health facilities in the two provinces detected 19 raw signals which was equal to the number of raw signals detected by primary schools, and which was 9.5 times of the number by pharmacies.
Daily average reported numbers of records per surveillance unit ranged from the highest to the lowest were, respectively, pharmacies, county hospitals, township hospitals, village clinics and primary schools. Varies existed within each of the five categories of surveillance units and also between Jiangxi and Hubei provinces. The numbers of reported records and rates of reported records per total outpatient volumes in health facilities in most cases were much higher in Jiangxi than those in Hubei province.
Cost per reported record from the five surveillance categories ranged from the highest to the lowest were, respectively, primary schools, county hospitals, township hospitals, village clinics and pharmacies. Cost per reported record in Jiangxi province in each of the five categories was higher than its corresponding in Hubei province.
Annual operation cost per surveillance unit ordered from the highest to the lowest were, respectively, pharmacies, county hospitals, village clinics, township hospitals and primary schools in Jiangxi province. The only difference in the order of annual operation costs per unit of the five categories between Jiangxi and Hubei was that in Hubei, the annual operation cost of a township hospital was higher than that of a village clinic. Annual operation costs per surveillance unit of the five categories in Jiangxi, again, were all higher than their corresponding in Hubei province.
When took a closer look at factors influencing costs, then time is a major resource for both the development and operation of the SSS. Time costs for data reporting, as variable costs, had the following characteristics: 1) at the initial stage of data reporting, more time was consumed compared with the time needed after familiarization with reporting. 2) daily reporting times varied within and across groups, but the median time for reporting in different groups was around 20 to 30 minutes with no big differences across groups. Together with the differences in annual salaries and daily total working hours, however, the opportunity costs of time in reporting data was observed to be extremely different within and across groups. 3) Most village doctors and around half of staff members from primary schools claimed to report data after normal work hours while others mainly worked during normal working time.
Time costs in data collection and reporting, daily management of data reporting and overhead costs allocated to the SSS were the main components of operational costs. If grouped by the categories of surveillance units, it became evident that village clinics incurred most of total costs of the SSS in each county.
Organizational approaches for daily management of data reporting, field trips for data management, etc. varied between Hubei and Jiangxi province. Huge differences on costs existed between Hubei and Jiangxi province, including costs for system development, training, daily data management, field trips, etc. Costs for signal verification were negligible in this study. This study found that a SSS which requires manual data reporting may generate more than small costs.
Potential Impact:
Infectious disease is one of the major challenges to human progress and survival for centuries. In order to mitigate potential damages, early warning of infectious disease epidemics has long been one of the fundamental tasks for infectious diseases surveillance and a primary concern for public health institutions. In the ISSC project, we developed and implemented a web-based syndromic surveillance system, called ISS system in rural China. Using a field experimental study design, we evaluate if it can improve the early detection of infectious disease epidemics under current conditions. The feasibility and acceptability of the system were also evaluated, as well as economic analysis. The ISSC project will have an impact at several levels in relation to an early warning surveillance system for infectious diseases.
On the level of overall public health, the critical role of an early warning system for disease outbreak has been demonstrated in disease prevention and control practices. Timely detection of infectious disease outbreaks can lead to timely and effective response activities, so that morbidity, mortality and unnecessary suffering as well as transmission to other areas or countries are minimized.
This study showed syndromic surveillance system could supplement traditional case report system in the early warning of infectious disease outbreaks, and both systems played an important and unique role in the capture of infectious disease outbreak signals. By comparing the signals between the syndromic surveillance and case report system, sensitivity and timeliness of syndromic surveillance were superior to those of case report system, but cases report system showed a higher positive predictive value. The further integration and combination of the two systems could augment the overall outbreak detection capabilities in rural China.
ISSC is also an innovative project. There has been practically no information on this research area (An integrated early warning surveillance system in rural areas) from China (or Asia). Although syndromic surveillance as an innovative tool for early detection of disease outbreaks has attracted a great deal of attention, most research in this field has been conducted in developed areas or countries where modern information technology infrastructure is well established and professional human resource support is available. Until the current study, there are no public reports on the development and implementation of a syndromic surveillance system in rural China. The ISSC project has filled a gap in knowledge here.
On the level of the individual patient and infected person, an early warning surveillance system can help to identify, confirm and treat patients and infected persons in a timely way, to avoid more serious adverse effects on them and their families, since emergent response systems are based on early warning surveillance systems. This is of great value for people, especially for rural people living in resource-limited environments who are most heavily affected by epidemics.
On the level of the health system and policy, developing, implementing and evaluating an integrated surveillance system which aims at the early detection of disease outbreaks will provide valuable evidence, experience and knowledge to policy makers. The development of the early warning surveillance system for infectious diseases is a great concern of Chinese government and is highly relevant to national policy and to program makers in China, as well as in other countries. With the strong political support we see a window of opportunity to develop and implement a syndromic surveillance system in rural China. The project will also provide data to policy makers on pre-clinical and clinical pre-diagnostic information in rural China. The different data streams used in ISS can help to explore the contribution of multiple data sources to early detection and the monitoring of infectious disease outbreaks.
Getting research into policy and practice is a major goal of our project. The project results showed local governments and their sectors demonstrated high political will and support both before and during the implementation of the ISSC, and the implementation seems technically feasible as a whole. From research project to practice, the cooperation of local government, public health departments and CDCs are the key to its success. Project information has been disseminated to policy makers and experts regularly during the implementation of ISSC, and they commented ISSC provides a classical example to improve infectious disease surveillance in rural China, and gives important public health implications.
In addition, we also found several constraints and challenges from the study. Electronic data is lacking for some data sources and manual labor is needed for data collection, which decreased the compliance of data collectors and increasing the difficulties of data quality control. A multi-sector cooperation and data sharing strategy between different data streams owners are also lacking, which made it challenging to collect syndromic data from different sectors. Despite the challenges regarding data collection and sharing, the ISS system showed significant advantages over the conventional surveillance system. With the growing availability of electronic data sources and the establishment of data sharing policy, the system will be more sustainable and timely responsive.
The main dissemination activities and the exploitation of results:
The dissemination of the results has followed two tracks: academic and policy maker (government) dissemination. The academic dissemination has been achieved by way of a number of scientific conference presentations coming out of the ISSC project. A number of publications have also generated from this project, and some manuscripts of the findings have been prepared or are currently under review. A list of conferences at which ISSC project outcomes have been presented is below, as a list of publications.
There have also been a number of meetings with policy makers from Ministry of Health, China CDC, provincial & municipal health authorities, provincial, municipal, county and district CDCs, Bureau of Education, Bureau of Food and Drug Safety throughout the life of the project, but particularly towards the end of the project.
The ISSC final Dissemination Conference titled as “The Development and Application of Syndromic Surveillance System in Rural China” was held in Shanghai on October 21-22 2014. More than one hundred participants from six countries and regions took part in the conference, including representatives from partner institutions, officials from the Public Health Agency of Sweden, European CDC, China CDC and the project administration office of EC granted projects; experts from different provincial CDCs in China; scholars from several Chinese universities, and National Institute for Communicable Diseases, South Africa, and R.D. Gardi Medical College, Madhya Pradesh, India. Representatives from Shanghai Medical Association, Shanghai Preventive Medicine Association and other civil society have also joined the conferences. The conference presented a comprehensive demonstration of the ISSC by covering the project designs, development of network platform, capacity building, global dissemination, fieldwork in China (Hubei and Jiangxi) and its outputs/outcomes.
Nine briefs of ISSC have been made and disseminated to policy makers and researchers during the project implementation. They have also been disseminated at different international and national conferences.