Existing benchmarks have been mainly designed for general-purpose single-processor architectures (such as workstations), for mainframes, and to some extent for vector processors. Commonly used performance measurements in mips or flops have little validity as an objective measurement for comparing the performance of different systems, while synthetic benchmarks such as Drystone and Whetstone are not intended to model the workload of embedded control and real-time applications. The BENCHMARK project aims to provide application-specific benchmarks for such applications, targeted at both uniprocessor and multiprocessor machines.
BENCHMARK's three main goals are to:
- characterise typical workloads for embedded control systems by a parameter set based on an examination and classification of applications
- define and develop benchmarks for embedded control systems
- define and develop an benchmarks for general-purpose multiprocessors.
The approach taken will be to categorise embedded control and real-time applications into different application classes. These classes will be thoroughly investigated in order to extract a workload parameter set representing the behaviour of the application programs (typical parameters might be locality, working sets, number of memory requests, etc). Workbenches and toolkits will be developed to assist in the generation of application program trace data, extraction of representative workload parameters out of trace data, and validation of the relevance of the workload parameters for different systems architectures. The specific characteristics of multiprocessor architectures will be the subject of a special study within the overall project. Finally, benchmarks for embedded control defined and established and real-time benchmarks for uniprocessors will be adapted for multiprocessor architectures.
The results of the project will allow a standardised, comparable and application-specific performance evaluation of different design alternatives for the giga-instruction/s processor addressed in project 7249, OMI/HORN.