Test Suites for NLP Applications

Project Information

TSNLP

Grant agreement ID: LRE62089

Project closed

Start date 1 December 1993

End date 1 August 1995

Funded under

• Specific programme of research and technological development (EEC) in the field of telematic systems in areas of general interest - Linguistic research and engineering -, 1990-1994

Total cost

No data

EU contribution

No data

Coordinated by

University of Essex
United Kingdom

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Exploitable results

Guidelines and a methodology for the construction of diagnostic data

The project involves defining guidelines and a methodology for the construction of diagnostic data (test suites) and designing and implementing related tools. The guidelines will be validated by constructing application-specific diagnostic data for French, German and English and testing the data on a number of applications ranging from parsers through grammar checkers to controlled language checkers. In addition to devising guidelines, the project investigates techniques and design and implement tools that will facilitate the construction, use and manipulation of test suites, such as a database for storing and manipulating test data, and tools for the (semi)automatic generation of test suites. An advantage of automation is that this allows dynamic test suite construction, where test data can be replaced by new data which tests the same phenomena. In this way, it may be possible to overcome one of the problems that sometimes crops up in system evaluation, namely that developers can tune their application so that it deals with static test data. The project has designed a methodology for the construction of test suites and produced a database of parallel test suites in three languages of some 4000 annotated items per language, with particular reference to three application types: parsers, grammar checkers and controlled language checkers. The test material is relevant to any developer or user of systems with grammar components who wish to test, benchmark or evaluate those systems. The test suites are fully documented and mounted on a database for ease of access and manipulation. Particular attention has been paid to the design and implementation of an efficient annotation scheme.