Final Report Summary - QI2S (Quick Image Interpretation System)
Executive Summary:
Next generation satellites need intensive (near) real-time on-board processing capabilities in order to fulfil demands of compute intensive image and signal processing workload.
Earth Observation Satellites (EOS) for hyperspectral imaging employ payloads, which typically sense radiation at wavelengths of 0.2 µm to 2.5 µm (UV-VNIR-SWIR), 3 µm to 5 µm (MWIR) and/or 8 µm to 14 µm (LWIR or TIR) and scan in up to thousands of pixels within one spatial dimension at once. Per spatial pixel hundreds of narrow bands (typically 5-50 nm wide) collect information about the radiance spectrum of each imaged pixel (typically sampling 100-900m2 on Earth). This enables image-based detection of various materials and objects through the identification of their unique spectral signatures. Such a strong concurrent differentiation regarding spatial and spectral information creates large signal data volumes at high-rate streams (e.g. the ASI-ISA agencies SHALOM hyperspectral project aims at imaging 200,000km2 per orbitday).
Most existing processors for space applications, such as Atmel AT697, Aeroflex UT699, Aeroflex Gaisler GR712RC and BAE Systems RAD750, provide performance levels below 1,000 MIPS, and are thus unsuitable for executing high-performance “next generation digital signal processing” (NGDSP) tasks in space missions.
In order to fulfil the demand for satellite on-board laborious image and signal processing, the Quick Image Interpretation System (QI2S) project aimed at implementing an innovative rad-hard, massively-parallel many-core computing system, augmented by specialized parallel processing software to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions. The objective of the QI2S project was to initially demonstrate the realisation of the future required level of satellite computation power for different on-board laborious data processing tasks. QI2S did so by innovating, developing and validating specialised (ITAR-free) technology for fast on-board Hyperspectral imagery processing.
The QI2S project has succeeded in proving the technological concept and demonstrated the potential to spaceborne on-board (near) real-time computing at performance-to-electric power ratio of approximately 15 GOPS/Watt or 2 GFLOPS/Watt. The project developed detailed specifications, implemented them through prototype hardware and software, demonstrated the solution via end-user led ground trials and drove the overall approach to pre-normative industry acceptance.
The QI2S prototype, that was designed, developed and validated, features multiple chip components of Ramon Chips’ innovative RC64 many-core architecture, embedded in a demonstration platform that includes a RC64 simulator, high-performance FPGA and parallel-processing application software developed to perform hyperspectral image processing algorithms.
The QI2S prototype platform enables for the first time, to assess performance of challenging parallel-processing software on simulated many-core hardware architecture. Hence, it is an important “road sign” that represents a measure to the potential performance of operational, full scale, RC64-based as well as other parallel-processing, many-core architectures for future spaceborne systems.
Addressing the challenge embodied by on-board near real-time processing of HS data also opens the opportunity to provide similar technology for a wide variety of high performance space borne computing.
QI2S represents a major step beyond ESA’s roadmap for the development of future payload processors.
The QI2S project reached a Technology Readiness Level (TRL) of 4 - technology validated in lab.
Project Context and Objectives:
Next generation satellites need intensive (near) real-time on-board processing capabilities in order to fulfil demands of compute intensive image and signal processing workload. Earth Observation Satellites (EOS) for hyperspectral imaging employ payloads, which typically sense radiation at wavelengths of 0.2 µm to 2.5 µm (UV-VNIR-SWIR), 3 µm to 5 µm (MWIR) and/or 8 µm to 14 µm (LWIR or TIR) and scan in up to thousands of pixels within one spatial dimension at once. Furthermore, per spatial pixel hundreds of narrow bands (typically 5-50 nm wide) collect highly precise information about the radiance spectrum of each imaged pixel (typically sampling 100-900m2 on Earth). This enables image-based detection of various materials and objects through the identification of their unique spectral radiance signatures. Such a strong concurrent differentiation regarding spatial and spectral information creates large signal data streams (e.g. the ASI-ISA agencies SHALOM hyperspectral project aims at imaging 200,000km2 per orbitday). The processing of such large data streams typically includes (representative for hyperspectral image processing algorithms, compliant with demands from European space agencies): radiometric correction for sensor-specific errors and radiometric calibration for upwelling radiance signals; cloud screening and atmospheric filtering, which computes the spectral reflection or emission profiles based on upwelling radiance and local features of the atmosphere; and automated information extraction algorithms, for specific material or object identification (e.g. anomaly detection, match filters, etc.). The resulting thematic maps and mosaics with different layers of extracted information can be rectified and geo-referenced to create a final information product for the end-user.
However, the potential of spaceborne hyperspectral imaging is far from being fully realized. One of the main bottlenecks is the long delay in response time, typically measured in weeks, caused by extremely large data volumes that need first to be transmitted to ground, then mobilized to the mission’s data exploitation centre, where they are prioritized, then submitted to effortful processing and only then distributed as useful (usually basic) products to the end-users. For example, a satellite orbiting at 680 km above earth having a ground speed of ca. 6,800 m/s, typically acquires imagery at rates of more than 2 Gbit/s, resulting e.g. in 20 Gbit of data for only one 70 km long ground image. Thus many terabits of data need to be downloaded at Gbit/s rates to facilitate near real-time hyperspectral data exploitation. Another reason why spaceborne hyperspectral imaging is far from being realized is of economical nature. Nowadays, most satellite compute systems rely on the use of hardware accelerators (i.e. FPGAs or ASICs) serving mostly a single purpose such as image compression. Even though FPGA accelerators are reconfigurable to some extent, it is very complicated and thus costly compared to designing or updating a software algorithm for an application specific or general-purpose processor (between which the developed RC64 and the QI2S prototype resides).
Most existing processors for space applications, such as Atmel AT697, Aeroflex UT699, Aeroflex Gaisler GR712RC and BAE Systems RAD750, provide performance levels below 1,000 MIPS, and are thus unsuitable for executing high-performance “next generation digital signal processing” (NGDSP) tasks in space missions. While NGDSP requirements are listed at 1,000 MIPS/MFLOPS, a more practical goal is 10,000 MIPS. Even the fastest, currently available space processor, SpaceMicro Proton200K, achieves only about 4,000 MIPS/900MFLOPS.
Recently, the US government has adopted Tilera’s Tile processor for use in space, in the framework of the OPERA program and the Maestro ASIC. Integrating 49 triple issue cores operating at 310 MHz, it is expected to deliver peak performance of 45,000 MIPS. Software development experience for the Maestro chip has encountered difficulties in parallelizing applications to the mere 49 cores of the Maestro. Some of the developments have underestimated the inter-core communication latencies involved in the tiled architecture of Maestro. Due to such difficulties, programmers are forced to cram multiple different applications into the many-core, resulting in additional difficulties regarding protection of each application from the other ones. Performance comparison for leading space processors versus year of introduction is plotted in Figure 1.
In order to fulfil the demand for satellite on-board compute intensive image and signal processing, the QI2S project aimed at implementing an innovative rad-hard, massively-parallel many-core computing system, augmented by specialized parallel processing software to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions.
Addressing the challenge embodied by on-board near real-time processing of HS data also opens the opportunity to provide similar technology for a wide variety of high performance space borne computing.
QI2S represents a major step beyond ESA’s roadmap for the development of future payload processors. Well-known ESA benchmarks consist mostly of basic signal processing algorithms like Fast Fourier Transform (FFT) and Finite Impulse Response (FIR) filtering and depend on the availability of flexible and scalable hardware and software solutions, since applications most likely change over space mission time and therefore space systems need to adapt within limited time frames.
Figure 1: Performance Comparison of RC64 Technology to Other Advanced Space Qualified Processors
2.2 Main Objectives
The principal objective of the Quick Image Interpretation System (QI2S) project is to initially demonstrate the realisation of the future required level of satellite computation power for different on-board laborious data processing tasks. QI2S does so by innovating, developing and validating specialised technology for fast on-board Hyperspectral Imagery processing. The development of a QI2S demonstrator is part of the effort to validate the usability and functionality of such system.
Thus the QI2S prototype concept implements an innovative rad-hard, massively-parallel many-core computing system and is augmented by specialized parallel processing software in order to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions.
The principal objective is further decomposed into the following QI2S technical development objectives:
OB1: Design, Develop and Validate QI2S many-core architecture
The foundation technology is Ramon Chips innovative RC64 many-core architecture – a custom high-performance 150 GOPS / 40 GFLOPS space qualified multiple-core parallel computing engine;
The QI2S many-core architecture is developed based on two unique architectural many-core design elements of Ramon Chips innovative RC64 many-core architecture –the Synchroniser/Scheduler and the shared memory system.
OB2: Develop QI2S technologies
QI2S leverages the QI2S Many-core architecture to research and develop the algorithms, the s/w building blocks, system software and the hardware platform. In particular the project developed, in addition to the many-core platform: QSBB – an application software implementing an interpreter with a finite number of specialized Building Blocks for HS data on-board processing and interpretation and: QMDL – a mission definition/command language, that will enable fast-turnaround reconfiguration of the processing procedure.
OB3: Integrate QI2S prototype
The QI2S project integrated a full QI2S prototype with the results obtained by achieving the previous objectives, namely the QI2S Many-core computing platform, the QI2S Software Building Blocks, the QI2S Mission Definition Language (QMDL), system software and related h/w integrated in a prototype with imagery and mission data interface to payload like links for imagery emulation, management & control.
OB4: Validate QI2S through ground-trials and simulation
The QI2S prototype was validated using a PC based imagery emulator & controller. Whose key The QI2S project validated the integrated prototype as well as the underlying architecture, components, QSBB and QMDL through a range of ground-trials. The QI2S validation process was performed at several levels, on the basis of real application scenarios, correlated to the development plan.
The QI2S consortium brings together 6 partners from 4 countries. The most part of the partners has successful past experience of mutual collaboration projects. Each partner has its own unique expertise in the individual domain which they have been assigned to for this project. The partners have built a strong and balanced consortium, including academic partners, research centres, large industries and SMEs.
Participant no. Participant organisation name Participant short name Country
1 [Co-ordinator] Elbit Systems Electro-optics ELOP Ltd. ELOP Israel
2 Digital Signalprocessing & Information Technology DSI GmbH DSI Germany
3 Compagnia Generale per lo Spazio CGS Italy
4 Ramon Chips RC Israel
5 ARTTIC SAS ART France
6 Technical University Braunschweig TUBS Germany
2.4 Project Structure
The work plan for the QI2S design, development, integration, test & validation project was divided into 5 technical Work Packages (WP). The WPs were further decomposed into specific tasks. For each task the leader and the contributors from among the consortium members were defined. A leader has been also nominated at WP level. Deliverables were defined for each work package.
Project Duration: 33 Months
Figure 2: Dependencies between Workpackages
Project Results:
3 Main S&T Results
3.1 Overview
The QI2S project has succeeded in proving the technological concept that combines a massively-parallel, many-core compute-hardware architecture, and an enabling parallel-computing, image processing software, leveraging a custom programming model, to demonstrate the potential to spaceborne on-board (near) real-time computing at performance-to-electric power ratio of approximately 15 GOPS/Watt or 2 GFLOPS/Watt.
The project developed detailed specifications, implemented them through prototype hardware and software, demonstrated the solution via end-user led ground trials and drove the overall approach to pre-normative industry acceptance.
The QI2S prototype, that was designed, developed and validated, features multiple chip components of the RC64 chip, embedded in a demonstration platform that includes a RC64 simulator, high-performance FPGA and parallel-processing application software developed to perform hyperspectral image processing algorithms.
Therefore, the QI2S prototype platform enables for the first time, to assess performance of challenging parallel-processing software on simulated many-core hardware architecture. Hence, it is an important “road sign” that represents a measure to the potential performance of operational, full scale, RC64-based as well as other parallel-processing, many-core architectures for future spaceborne systems.
3.2 The RC64 innovative architecture
The RC64 innovative chip architecture which is the basis for the QI2S development is briefly described herein below:
The RC64, is a novel rad-hard 64-core signal processing chip, which targets DSP performance of 75 GMACs (16bit), 150 GOPS and 20 single precision GFLOPS while dissipating less than 10 Watts. RC64 integrates advanced DSP cores with a multi-bank shared memory and a hardware scheduler. The programming model employs sequential fine-grain tasks and a separate task map to define task dependencies.
RC64 is currently being implemented in the framework of another project, as a 300 MHz integrated circuit on a 65nm CMOS technology, assembled in hermetically sealed ceramic CCGA624 package and qualified to the highest space standards. This new package and new process technology enable the inclusion of twelve integrated full duplex high speed serial links (HSSL) using CML SERDES interfaces on chip at 2.5Gbps rate each, with an aggregate 60Gbps throughput. Several protocols such as SpaceFibre are considered for HSSL, aiming at efficient connectivity among multiple RC64 chips and other FPGAs, ASICs. Also, faster and denser DDR3 SDRAM interface is made possible. Reed-Solomon ECC is employed to protect from DDR2/3 SEFI and SEE. The 32-bit wide DDR2/3 interface supports up to 25Gbps throughput. Other I/O interfaces in RC64 include SpaceWire for control, Parallel LVDS interfaces for advanced ADC and DAC devices connectivity and interface to flash memory.
The on-chip shared memory system of RC64 is based on each core having its own write-through data cache, an instruction cache, and a private store, supporting the unique task-oriented programming (TOP) model. All cores access the single shared memory with 256 ports and a 64-to-256 ports multistage interconnection network, enabling simultaneous access of all processors to shared memory with very little conflicts.
The on-chip 4MByte shared memory acts as a local-store memory. Access to off-chip DDR2/3 memory is facilitated by software-controlled DMA. This approach simplifies software development and it is found to be very useful for DSP applications, which favour streaming over cache-based access to memory.
The RC64 Core firmware architecture is depicted in Figure 2 and the concept of operation of the Many-Core is shown in Figure 3. A central scheduler assigns tasks to processors. Each processor executes its task from its cache storage, accessing shared memory only when needed. When task execution is done, the processor notifies the scheduler, which can subsequently assign a new task to that processor. Access to off-chip streaming channels, DDR2/3 memory, and other interfaces happens only via programmable DMA channels. This architecture and concept of operation have been thoroughly simulated and realized to the extent detailed herein below by the QI2S prototype. When a single RC64 has insufficient processing power for the application, multiple RC64 chips can be accommodated, as shown in Figure 4. The four chips are interconnected with high-speed channels. RC64 has been designed for integration with tens or hundreds of other RC64 chips, enabling very powerful digital signal processing in space.
The future, full-scale RC64 chip-based parallel-computing platforms will be customized platforms that will enable scalable performance using multiple chips communicating and processing concurrently, which can reach 10's of GFlops and 100's of GOPS.
Figure 3: The RC64 Core architecture
Figure 4: RC64 Many-core concept of operation
Figure 5: Single board DSP using 4 interconnected RC64 chips
3.3 The QI2S prototype implementation
The foundation technology is the RC64 Many-core – a custom high-performance 150 GOPS / 20 GFLOPS space qualified many-core parallel computing engine; QSBB – a software implementing an interpreter with a finite number of specialized building blocks for HS data processing and interpretation; QMDL – a mission definition/command language interpreter, that enables fast-turnaround reconfiguration devoid of elaborate time consuming testing attributed today to spaceborne payloads operational s/w change procedures.
Thus the QI2S prototype concept implements an innovative rad-hard, massively-parallel many-core computing system and is augmented by specialized parallel processing software in order to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions. The development of a QI2S demonstrator is part of the effort to validate the usability and functionality of such system.
However, since the full scale RC64 chip is still under development in the work frames of other projects, the QI2S technology is implemented in this project with a downscaled FPGA board. The Image processing System (IPS) of the QI2S is integrated in the ASIC prototyping engine DNV7F1A from DiniGroup. It contains a high speed FPGA and as a stand-alone solution it can be integrated in a special housing containing power supply, fan, connectors, etc. The offered interfaces include among others Ethernet connectors and SFP+ interfaces for high data rates. A DDR3 UDIMM socket enables up to16GB of standard, off-the-shelf memory to be used by the FPGA. The final configuration of the DNV7F1A board used in this project contains:
▪ Xilinx Virtex 7 FPGA (7VX690T-1) with speed grade 1.
▪ 8GB DDR3 memory
▪ Chassis including power supply
The block diagram of the IPS is shown in Figure 5.
Figure 6: Block Diagram of the Image Processing System (IPS)
In the IPS, a hardware (task) scheduler dynamically allocates, schedules, and synchronizes tasks among the parallel processing cores according to the program flow. Hence, it reduces the need for an operating system (OS) and eliminates large software management/execution overhead. No OS is deployed to the cores. The complementary task oriented programming model (TOP) reduces efforts on parallel programming and is based on the C programming language, adding some few extensions. One of these extensions comprises of a so-called “Task Map” that explicitly embodies the program flow (i.e. task-dependency graph) and enables efficient scheduling of software tasks using the hardware scheduler, thus preventing software from relying on shared memory synchronization and eliminating the need to manually schedule tasks to processors.
The many-core computing platform is integrated with a software interpreter with a finite number of finely designed software building blocks for fundamental processing and interpretation of hyperspectral imagery.
The QI2S Software Building Blocks (QSBB) represents the algorithms that are used for image interpretation. There are three QSBBs arranged in a consecutive order (Figure 7). The intermediate result can also be accessed for reading to offer debugging capabilities.
These building blocks are designed to form the fundamental toolbox comprising essential mathematical operations and filters which accommodate radiometric correction, atmospheric correction and materials/objects detection by identifying characteristic spectral signature features within a certain spectral range and threshold:
▪ Radiometric Calibration
The raw image from the hyperspectral camera is processed for authentic radiometric correction (i.e. for a real imager), which is separated into six sub blocks. This stage is required to correct the uniformity between the micro sensors for each spectral band and each pixel on the vertical row of sensors. In addition this algorithm handles the rotation registration and binning issue to create the exact number of bands required to the analysis of the spectral domain.
▪ Atmospheric Correction
This stage with five sub-blocks is required to correct the atmospheric effects that were added to the sensed image on the optic flow between earth and satellite. In addition, this algorithm handles the transformation of images from radiance units to those of reflectance. The algorithm uses two passes to reduce the two degrees of unknown water vapour, and the visibility range.
▪ Material Detection
The last stage in the image interpretation is the material detection, which consists of four sub-blocks. The output of the algorithm is a grey scale level for every pixel representing the probability of the material presence. A threshold transforms the grey scale result in a binary result. The three materials that the implemented algorithms detect are: green vegetation, clay minerals and plastic greenhouse. Cloudy pixels or no detection is indicated as well. In hyperspectral imagery, different processing procedures that usually make use of similar mathematical functions but use different parameters, variables, thresholds and apriori data, need to be applied for extraction of different types of information from the same data set.
Figure 7: Data Flow of the Image Processing System
A PC based system named Emulator & Controller (EC) represents the EO payload interface, emulating hyperspectral Imagery and performing management, control and monitoring of the IPS. Aside the IPS, the EC is also a part of the QI2S prototype. The main features of the EC are:
• Emulate: Row-wise streaming of hyperspectral images via a high speed link to the IPS.
• Control:
▪ Mode management and application control
▪ Work flow and operation configuration via the QMDL software
▪ Transferring of pre-calculated mission support data (tables) to the IPS, Start and stop IPS processing
• Analyse:
▪ Reading and analysing the correctness of the data and presents the relevant timing measurements
▪ Request and interpret BIT of the IPS, verify intermediate results and record all kinds of inputs and outputs
The EC is equipped with a high-speed serial card (Xilinx KC705) to emulate the payload with a realistic data rate of 2 Gb/sec (Figure 8). To enable the controlling, emulating and analysing capabilities of the EC the test software package GSEOS V is used. GSEOS V is designed to support all stages of experiment development, from bench checking and spacecraft integration up to “quick-look” during flight operation. It supports testing of units under near real-time conditions by using a data-driven concept in contrast to less efficient¬ polling. It provides response times of less than 10 μs. It is user configurable and therefore very flexible. GSEOS V is able to:
• Interpret the data to be transferred from EC to IPS (e.g. digest the QMDL Config Packet or Image File)
• Send those data to the IPS.
• Control the IPS to enable several work flows (normal processing / verification).
• Read, analyse and display the outputs of the IPS.
• Store all the inputs and outputs for future reference.
Figure 8: QI2S Demonstrator (block diagram)
These procedures must be tailored by simple mission commands and therefore a flexible Mission Definition Language (QMDL) is also developed, based on the GSEOS V software package on the EC side and on interpreting s/w on the IPS side, to enable fast-turnaround reconfiguration of the QI2S hyperspectral imaging interpretation process, setting different processing chains for different hyperspectral detection scenarios.
3.4 Performance Assessment for a RC64 Full-scale Chip
The developed QI2S application software runs on three platforms:
1. Software emulator
2. Software simulator
3. FPGA platform
However, none of these platforms achieves a cycle-accurate representation of executing QI2S application on the target, full-scale architecture of the RC64, as follows:
1. The software emulator executes the code on Intel X86 architecture. Tasks are executed sequentially. I/O is only roughly estimated. Memory constraints are ignored.
2. The software simulator employs Tensilica DSP cores that are differ from RC64 DSP cores produced by CEVA.
3. The FPGA platform emulates only 16 Tensilica RISC cores executing at 35MHz, performing 560 MFLOPS and provides only a partial simulation of the rest of the RC64 components and I/O. The full-scale RC64 operates 64 VLIW CEVA cores executing at 300MHz, performing up to 38GFLOPS.
The performance estimation for a full scale RC64 platform is based on analysis of the computational bottlenecks of QI2S. This application consists of floating point operations. Assuming complete parallelization, such that, each computational step is parallelized in a balanced manner to all 64 cores, we note that higher performance is possible if some or all computations are converted to fixed-point arithmetic.
Measurements of floating point computations counted on the emulator shows that 1000 image “rows” (1 “row” = 1000 spatial x 320 spectral pixels) require some 360M floating point operations. 93-94% of all floating point computations are spent for interpolation methods. The measurement reliably represents actual computations, regardless of the architecture on which QI2S is executed. The measurement assumes that there are no I/O bottlenecks and no memory congestion. This is a reliable assumption for a well-optimized application.
RC64 is designed for a performance of 38 GFLOPS (Giga floating point operations per second). Thus, the RC64 estimated performance based on the QI2S results is:
This is an upper bound on performance of a single RC64 chip when all arithmetic employs floating point operations. Expected actual performance in floating point is somewhat lower because:
1. Non-arithmetic parts of the code, including iterations and other instructions, are not counted. Typically, these are optimized so that they do not affect performance too much. However, this analysis has not been performed.
2. The reporting counting has not been validated by a third party. It is possible that correcting errors in measurement may reduce estimated performance by a small margin.
When pixel rates higher than 34 Mpixels/second are required, multiple RC64 chips may be employed. The image data can be divided into several RC64 chips operating in parallel. For instance, consider a hyperspectral imaging LEO satellite where each pixel’s GSD (ground sample distance) is 10×10 meter (e.g. the hyperspectral payload designed for the ASI-ISA agencies SHALOM mission), the imager swath spans 1000 pixels, and the satellite ground velocity is approx. 7,000 meters/second. The resulting synchronous scanning pixel rate is 224 Mpixels/second, and 7 units of RC64 are required to handle this data rate. A more realistic asynchronous pixel rate of 75Mpixel/seconds (due to both FPAs readout limitations and even more important, due to “back-slew” motion compensation maneuver required to increase signal integration time) would require only 3 units of RC64 for on-board, real-time processing of hyperspectral imagery!
Converting floating point computations into fixed point should result in even faster processing rate. Indeed, considering that the pixel image data is typically digitized into 12 to 16 bits, 16-bit arithmetic may suffice for most computations. The fixed point peak performance of RC64 is 150 GOPS, 4 times faster than FLOPS rate. Thus, the estimated data rate of a single RC64 in fixed-point computation is 135 Mpixels/second. This one-RC64 data rate is more than sufficient for the hyperspectral camera imagery stream example mentioned above.
Potential Impact:
4 The potential impact
4.1 Socio-economic impact and the wider societal implications
Building upon the innovative technologies described above, the QI2S prototype development results, initially demonstrate and pave the way to an operational QI2S that will enable significant reductions of delay in the delivery of Earth Observation Hyperspectral processed/interpreted data to the end user, from days or weeks to real-time or near-real-time by designing, developing, integrating and validating a full scale platform.
The need for spaceborne hyperspectral remote sensing emerges dramatically and is driven by a continuously increasing range of important applications in various fields. Many evolving applications require fast imagery data exploitation to support real-time decision-making. For example the cases of monitoring evolving incidents and disaster analysis: e.g. for fighting wild fires (real-time information on expected spreading directions based on forest biomass analysis and humidity conditions is highly desired), or furthermore for search & rescue in case of accidents in remote or maritime locations and scenarios. Such rescue operations demand immediate detection of concealed or drifting debris, survivors or bodies. Additionally, water contamination, air pollution, atmosphere conditions monitoring and precise agriculture (disease/stress precise detection, customized fertilization etc.) are also some of the many fields that can significantly benefit from real/near real-time spaceborne hyperspectral imagery data exploitation which is accessible and affordable.
QI2S helps usher in the next generation of lightweight HS satellites able to provide a complete suite of on-board imagery processing and information extraction by automated algorithms, for day-to-day operations, such as oceanographic mapping, as well as for evolving incidents, such as post disaster analysis.
QI2S responds to the need for EU to solve Societal Challenges and to the need for space companies to remain competitive by creating and selling successful services and space infrastructure that contribute in solving these challenges. Hyperspectral missions and QI2S can provide precious information to the society allowing Earth monitoring in real time. QI2S on board of new space missions will allow increasing our knowledge related to our Earth, helping us to better understand ourselves and our environment.
Spaceborne on-board (near) real-time imagery processing opens not only the opportunity to provide HS “avant-garde” services from space but also provides for similar technology to serve a variety of other high performance spaceborne remote sensing computing applications such as SAR, on-line image data compression, real-time EO object detection & recognition, real-time image rectification and geo-location, etc. Hence, it will open the space sector to a wider range of new applications and also to a new way of delivering space services matching evolving application requirements to broader public communities and potentially to new user markets.
4.1.1 Competitiveness
The product
Following this vision, two main assets have been individuated for QI2S market opportunities:
1. Onboard Processing Platform Exploitation: this technology will be able, in the future, to reach performances of hundreds of GOPS and tens of GFLOPS. The development of this disruptive, key-enabling technology will radically innovate the on-board processing units, compared to existing on board hardware solutions, and will increase on-board compute performance capabilities by 10-100x. The hardware platform is demonstrated for real-time on-board analysis of hyperspectral data, but can be used on a wide range of different processing performed directly in space. Moreover such design can simplify the instrument design, itself leading to reductions in mass, power and volume for missions as remote sensing where multiple instrument components necessitate optimization of these resources. In this way, strategic application of information technology advances can lead to measurable improvements in both instrument data reduction and design. From this point of view QI2S product will substantially help accelerate and usher in the next generation of lightweight, low-cost Hyperspectral EO satellites.
2. EO Products Fast Delivery Service Exploitation: The need for innovative EO services based on spaceborne hyperspectral remote sensing services emerges dramatically and is driven by a continuously increasing range of important applications in various fields. Many EO services require fast imagery data exploitation to support real-time decision-making or to provide the citizens with up-to-date information for our daily life. Our vision is to realize future earth observation missions capable of delivery EO products directly to the user at each satellite pass. The reception of the satellite products has not to be so heavy. The data processing on board reduce the amount of data need to be downloaded and facilitate real-time hyperspectral data exploitation.
The QI2S exploitation is unique in the field of future EO mission because it blends fully the hardware development for disruptive on board processing platform and the on-board image processing, which support all key aspects of real time EO service delivery. QI2S has been tested in laboratory (TRL 4) through the WP 5 activity. This technology will be able, in the future, of reaching very high performances, although for demonstrational purposes it is now implemented on a downscaled environment.
The Market
A large number of EO satellites are expected to be launched in the next years. As reported by Euroconsult in “Satellite-Based Earth Observation” Market prospect to 2022, about 180 new EO satellites will be launched over 2013 to 2022. Assuming that each one needs a framework for on-board downstream applications, then the global market is 180 units per year on average over the next 10 years. Supposing a cost of €5M per system, this could represent a global market of € 900M in 10 years.
Growth expected in the market of space payload systems as QI2S is based on two processes:
• On-board Processing Platform: enabling novel missions and applications that were thus far impossible in space. The growth rate of adoption of this new technology is expected to be very fast, much faster than the base growth in the space industry.
• EO Products in real time: the transition of computations from ground stations to space, in order to accelerate time-to-results by eliminating the need to transmit high data volumes from space to ground;
Target Stakeholders
The target stakeholders for QI2S range from important Space enterprises, SMEs and institutions at European and international level. The principal entities, which can be interested in the outcomes of QI2S project are:
• Governmental institutions,
• Space Agencies,
• Large and medium companies active in the Aerospace domain
• Large and medium companies active in the EO services domain
About EO services each stakeholder can have different needs related to the image characteristics. The mission design and the sensor definition will have to take into account this variability providing a camera with acquisition capabilities able to meet the full spectrum of them, both civil and government entities requirements. As a possible set of user applications it is worth to mention:
• Worldwide complex emergencies for situation awareness and rapid damage assessment;
• Natural disasters (floods, earthquakes, volcanic activity, tsunami, drought, vegetation fires, etc.);
• Homeland security and intelligence applications: surveillance and reconnaissance;
• Technology risks, major transport routes, accidents associated with transport, industrial activities.
About the Large and medium companies active in the Aerospace domain they have a great interest on the Onboard Processing Platform asset, while the Large and medium companies active in the EO services domain are concentrated on the EO Products Fast Delivery Service.
Potential Competition
The future exploitation of QI2S leans on two assets: EO Products Fast Delivery Service asset and On-board Processing Platform asset. It has to be taken into account that the EO Products Fast Delivery Service is a new technological frontier that goes beyond competition. Before having a real competition in this asset, the On-board Processing Platform has to be quite ready to be launched.
QI2S has developed a very innovative idea, where a first approach to Hyperspectral Products Fast Delivery Service has been developed at prototype level. This approach will allow the QI2S team to be ready to deliver new added value products, processed directly on board, once the processing platform will be ready for space. Currently the completion for such system can be analysed focusing on the On-board Processing Platform asset, where the industrial and scientific community is concentrated on.
While the commercial processors have short life and are quickly replaced by new generation architectures, in the space sector processor performance research goes hand in hand with reliability. Moreover in this sector, is usual to have long-time programs that require solutions supported in the long term (up to 15 years). This entails, of course, a chronic delay of the space sector compared with the SOTA technology on the commercial and industrial market.
The computing demands and the low power requirements of current sophisticated space missions are starting to exceed the performance capabilities delivered by state-of-the-art flight qualified processors, so the high performance of the future RC64 multi-core processor, becomes an innovative strength in QI2S project. In this project the research activities have been carried on and the processor has been adapted for Image Processing System.
An overview of the competition in space processors (current and future processors) is given in Table 1. All the listed processors are or will be RAD-HARD.
Processor Manufact. year Freq,
MHZ MIPS MOPS MFLOPS Power cons. [watt] Watt/
MFLOPS
RC64 Ramon Chips 2016 300 150,000 38000 8 2.1 x 10-4
Proton200k Space Micro, Inc. 2008 4000 900 4,24 46 x 10-3
Rad5545 BAE 5200 3700
RAD750 BAE 2001 200 266 5
NGMP-LEON4 Aeroflex-Gaisler Future (2020-2025) 150 75 x 10-3
GR712RC-LEON3 Aeroflex-Gaisler Future- later 2015 100 200 DMIPS
[140 DMIPS/core]* 200 MFLOPS 1,5
UT700-LEON3 Aeroflex-Gaisler 2014 166 1.2 DMIPS / MHz [233 DMIPS/core]* Max 4
UT699-LEON3 Aeroflex-Gaisler 2011 60 75 DMIPS [92 DMIPS/core]* 5.5 4 x 10-2
AT697F-LEON2 ATMEL 2009 100 86 DMIPS 23 MFLOPS 1
AT6981-CASTOR Future- later 2015 200 150 MIPS 40 MFLOPS 36 x 10-3
OPERA MAESTRO
(ip removed) Tilera-EZ Chips Future- later 2015 350 5000 MFLOPS 18 W 52 x 10-3
*With latest compiler, 0 wait states on SRAM and cache enabled
Table 1: Space Processors Performance Comparison
In the Europe, the main manufacturer of processors to space applications is the Swedish Gaisler Research (acquired in 2008 by Aeroflex Inc), a designer of rad-hard IP (semiconductor intellectual property core) for space applications, including the open source LEON processor.
The new company, Cobham Gaisler AB, (Aeroflex-Gaisler) provides IP cores and supporting development tools for embedded processors based on the SPARC architecture and it is specialized in digital hardware design for both commercial and aerospace applications.
Several versions of the LEON processor have been developed by Aeroflex Gaisler:
• LEON2 based processors
• LEON3 based processors
• LEON3 FT- A fault-tolerant version of LEON3
LEON3 FT - a fault-tolerant version of the standard LEON3 SPARC V8 Processor
Aeroflex Gaisler provides 3 different microprocessors LEON3 FT based:
• UT699-LEON3 is a pipelined monolithic, high-performance, fault - tolerant SPARC TM V8/LEON 3FT Processor. The UT699 provides a 32-bit master/target PCI interface, including a 16 bit user I/O interface for off-chip peripherals. The clock frequency is 60MHz and achievefrequency is 60MHz and achieves the performances of 92 DMIPS/core. Its Power Consumption is 5,5 W.
• UT700-LEON3 features a seven stage pipelined monolithic, high-performance, fault-tolerant SPARC V8/LEON 3FT Processor. The clock frequency is 166MHz and integer performance is 1.2 DMIPS / MHz. Its Power Consumption is 1,5 W.
• GR12RC-LEON3 a dual-core LEON3FT 32-bit processor, with two fully SPARC V8 compliant integer units and two high performance fully pipelined IEEE-754 floating point units. The clock frequency is 100MHz and achievefrequency is 100MHz and achieves the performances of 200 DMIPS and 200MFLOPS. Its Power Consumption is 1,5 W.
PROCESSORS WITH PERFORMANCE COMPARABLE TO LEON3
BAE System Electronic Inc. realized the RAD750 that was released in 2001, with the first units launched into space in 2005. The CPU has 10.4 million transistors, nearly an order of magnitude more than the RAD6000 (which had 1.1 million). It is manufactured using either 250 or 150 nm photolithography and has a die area of 130 mm2. It has a core clock of 110 to 200 MHz and can process at 266 MIPS or more. The CPU can include an extended L2 cache to improve performance.
The CPU itself can withstand 200,000 to 1,000,000 rads (2,000 to 10,000 gray), temperature ranges between –55 °C and 125 °C and requires 5 watts of power. The standard RAD750 single-board system (CPU and motherboard) can withstand 100,000 rads (1,000 gray), temperature ranges between –55 °C and 70 °C and requires 10 watts of power.
Another similar product is Proton 200k manufactured by Space Micro Inc. (USA). It is a high-speed, SEE-hardened DSP available for space and satellite applications available from 2008.
The main manufacturers involved in electrical component to space applications are developing a new generation of processors, available in the next future. The new LEON4 core is an evolution from the LEON3 core with improved performance thanks to wider internal buses, modified pipeline and support for a Level-2 cache. A LEON4 evaluation board with a dual-core processor system is currently available.
The GR-CPCI-LEON4-N2X evaluation board has been designed for evaluation of the Aeroflex Gaisler LEON4 Next Generation Microprocessor (NGMP) functional prototype device. The NGMP functional prototype is a system-on-chip with four 32-bit LEON4 SPARC V8 processor cores connected to a shared 256 KiB Level-2 cache and several high-speed interfaces, including an 8-port SpaceWire router and dual gigabit Ethernet interfaces. The architecture provides improved support for debugging and software partitioning together with extended support for both symmetric and asymmetric multiprocessing.
The Atmel AT6981 Castor device is a LEON2-FT based system-on-chip with multiple integrated peripherals including an eight-port SpaceWire router and three internal SpaceWire engines each containing three DMA channels, an RMAP initiator, and an RMAP target.
The RAD5500 family of radiation-hardened processors use the QorIQ Power Architecture with processor cores based on versions of the Freescale Technologies e5500 core. The RAD5510, RAD5545, and RADSPEED-HB (Host Bridge) are three systems on a chip processors implemented with RAD5500 cores produced with 45 nm SOI technologies from the IBM Trusted Foundry.
The RAD5545 processor employs four RAD5500 cores, achieving performance characteristics of up to 5200 MIPS and over 3700 MFLOPS.
Based on the RAD5545, the RADSPEED-HB is intended for host processing and data management support for one to four RADSPEED DSPs. The RADSPEED-HB replaces a secondary DDR2/DDR3 memory interface connection found on the RAD5545 with connections for RADSPEED DSPs instead. (Note that RADSPEED DSPs are entirely different processors that are specialized for digital signal processing and are not to be confused with the RADSPEED-HB, which serves as a host bridge.)
The Maestro processor was designed based on the TILE64 processor. It has 49 cores arranged in 7 by 7 meshes on a chip. It also added an FPU in each core for high performance for floating point operations. Since the Maestro processor was designed for space applications, the processor has been radiation-hardened by design, which resulted in reducing the number of cores from 64 to 49. The Maestro processor runs at up to 350 MHz clock frequencies. On the Maestro processor, we implemented an FFT and an image processing application called CRBLASTER and evaluated the performance. The FFT is a well-known and commonly used signal processing kernel. CRBLASTER is a parallel-processing image-analysis application that does cosmic-ray rejection on CCD (charge-coupled device) images using the embarrassingly-parallel L. A. COSMIC algorithm. Both applications were written in C. CRBLASTER uses the high-performance computing industry standard Message Passing Interface (MPI) library. The achieved performance of the FFT was up to 3,813 MFLOPS, and the speedup compared to single tile was 46.4 using 49 tiles. The speedup for CRBLASTER, which was memory-bound, was up to 12.5 using 36 tiles.
Potential Risks
The main identified risk for implementing our QI2S individual exploitation plan lies in the low level of maturity of the system. The TRL achieved at the end of the project is 4 (validated in laboratory). This level of technology readiness is far from to be ready to use QI2S in future space missions, if the research on this technology will not be continued to reach a higher level of maturity.
4.1.2 QI2S dissemination activities
QI2S initially envisaged as part of Work Package 6, T6.1 “Dissemination activities” to establish at project start a detailed dissemination plan (including communication strategy and contributions). Although defined and coordinated by the leader of T6.1 partner ARTTIC, this plan was to be agreed upon by all project partners, which would have to implement its actions along the project duration. This plan aimed at defining the target audiences, objectives, communication channels, tools and support material, activities, responsibilities, timing, budget allocations. It also was to select dissemination actions to implement according to the plan.
The dissemination plan was established and agreed upon during the project first year. It focused on the following target groups and objectives:
Target Group Objectives
Primary target group - Industry Raise awareness to the technology developed by QI2S solution and the different possibilities it can create for onboard EO image interpretation systems
Secondary target group – Academy Promote research on hyperspectral imaging, promote the use of hyperspectral imaging in research in other topics such as environment studies, agriculture, space research
Tertiary target group - General Public Raise awareness in the public regarding hyperspectral imaging, and EO image interpretation.
The table below shows the overview of the scheduled actions in order to reach these targets:
Activity Project Month
Project presentation materials (Project logo, project PPT, document templates M3
Public website M4
Printed materials (Project Brochure, poster) M12-M24
Scientific Publications M1-M33
Participations in conferences and events M1-M33
Project dissemination event M33
The project communication materials
The QI2S Project dissemination strategy relied on diverse communication materials. It included the items listed below.
• The project logo was drawn at early stages of the project. A few options were presented to the consortium member who decided on the one which reproduced on every QI2S document, including this deliverable (at the first the page). The project identity was then built around this logo, its spirit and colours. It resulted in the production of the project presentation / communication templates to be used by all partners when communicating about QI2S.
• The project printed material covered a standard brochure and Poster. Their finalisation and printing were delayed due to the fact that the dissemination team waited for promising technical results to be included in their content. They were distributed to partners to support their speech at conferences at mid-project.
Figure 9: Project Logo
Figure 10: Project Brochure
Figure 11: project poster
QI2S publications and participation in conferences and events
During the course of the project and along project findings and promising results, partners made research developments and outcomes available to the scientific community through 2 scientific publications, mainly in peer-reviewed journals. On top of that partners has participated in several events and represented QI2S project. The list of publications and events are available in section 4.2.
QI2S dissemination event
The consortium planned a dissemination event hosted by CGS in order to present the project's results to stakeholders in the industry. Unfortunately not long before the event the QI2S system, still in a mature stage suffered from technical problems which did not allow the consortium to present it in public. Due to the summer vacation and the fact that the project has ended quite after it, the consortium was not able to plan another dissemination event and we were forced to let go of this activity in order to put our efforts on final R&D actions.
Collaboration with other FP7 projects
Due to the fact that many members of QI2S consortium are also partners in the ongoing project MacSpace (607212), a close communication and collaboration was formed between the two projects. MacSpace will continue after QI2S has finished and will potentially exploit knowledge created in QI2S project by including additional work on the hyperspectral application. Some dissemination activities were shared between the two projects since they involved and presented both projects, especially those made by RC who is the coordinator of MacSpace project.
Future dissemination activities done by MacSpace will also potentially include information about the QI2S project, and therefore will continue dissemination about the QI2S project.
List of Websites:
4.1.3 QI2S public Website
The concept for the implementation of the public website was defined as follows:
Target audience Large audience ranging from experts from industry and research, scientists community, users of the knowledge and technologies resulting from this research work, and stakeholders, policy makers, European Commission as well as the general public.
Communication channel The domain name : http://www.qi2s.eu/
Objective Support dissemination of information about QI2S.
Present the main benefits of QI2S regarding Earth Observation (EO) Hyperspectral (HS) technologies.
Expected impact Enhance the project visibility, create awareness of the major scientific and technological achievements and contribute to creating a positive view of the European but also world-wide community of equipment stakeholders to enable their adhesion and possible contribution to the QI2S developed standards.
Content Section Content
Home Page Project summary
News and Events The latest information about the project and events.
About the project State of the Art – Background, objectives, activities and expected results of QI2S project (Page will be updated along the project)
Partners Presentation of the consortium. Identification and short presentation of the partners as well as presentation of their role in the project, with the links to their own websites
Publishable Outputs Project publishable outputs identified under the following groups:
• Press Releases & Articles
• Downloads (Flyers for various events and dissemination activities)
• Publications (also includes attended events with presentations)
Page will be updated along the project
Links Links to related projects, FP7 sites, European sites
Contact us Contact details of the QI2S coordinator, and a contact form
Date(s) Preliminary version was defined M3. The web site was published internally in M4 for final review. Continuous updates are made regularly.
Leader CGS
Contribution • For the elaboration of the content, Project Office team first prepared drafts based on information available, and then contacted the partners for required contributions/modifications.
• Editing and quality check was organised by the PO team.
• Validation: Notification procedure organised by the PO team
Next generation satellites need intensive (near) real-time on-board processing capabilities in order to fulfil demands of compute intensive image and signal processing workload.
Earth Observation Satellites (EOS) for hyperspectral imaging employ payloads, which typically sense radiation at wavelengths of 0.2 µm to 2.5 µm (UV-VNIR-SWIR), 3 µm to 5 µm (MWIR) and/or 8 µm to 14 µm (LWIR or TIR) and scan in up to thousands of pixels within one spatial dimension at once. Per spatial pixel hundreds of narrow bands (typically 5-50 nm wide) collect information about the radiance spectrum of each imaged pixel (typically sampling 100-900m2 on Earth). This enables image-based detection of various materials and objects through the identification of their unique spectral signatures. Such a strong concurrent differentiation regarding spatial and spectral information creates large signal data volumes at high-rate streams (e.g. the ASI-ISA agencies SHALOM hyperspectral project aims at imaging 200,000km2 per orbitday).
Most existing processors for space applications, such as Atmel AT697, Aeroflex UT699, Aeroflex Gaisler GR712RC and BAE Systems RAD750, provide performance levels below 1,000 MIPS, and are thus unsuitable for executing high-performance “next generation digital signal processing” (NGDSP) tasks in space missions.
In order to fulfil the demand for satellite on-board laborious image and signal processing, the Quick Image Interpretation System (QI2S) project aimed at implementing an innovative rad-hard, massively-parallel many-core computing system, augmented by specialized parallel processing software to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions. The objective of the QI2S project was to initially demonstrate the realisation of the future required level of satellite computation power for different on-board laborious data processing tasks. QI2S did so by innovating, developing and validating specialised (ITAR-free) technology for fast on-board Hyperspectral imagery processing.
The QI2S project has succeeded in proving the technological concept and demonstrated the potential to spaceborne on-board (near) real-time computing at performance-to-electric power ratio of approximately 15 GOPS/Watt or 2 GFLOPS/Watt. The project developed detailed specifications, implemented them through prototype hardware and software, demonstrated the solution via end-user led ground trials and drove the overall approach to pre-normative industry acceptance.
The QI2S prototype, that was designed, developed and validated, features multiple chip components of Ramon Chips’ innovative RC64 many-core architecture, embedded in a demonstration platform that includes a RC64 simulator, high-performance FPGA and parallel-processing application software developed to perform hyperspectral image processing algorithms.
The QI2S prototype platform enables for the first time, to assess performance of challenging parallel-processing software on simulated many-core hardware architecture. Hence, it is an important “road sign” that represents a measure to the potential performance of operational, full scale, RC64-based as well as other parallel-processing, many-core architectures for future spaceborne systems.
Addressing the challenge embodied by on-board near real-time processing of HS data also opens the opportunity to provide similar technology for a wide variety of high performance space borne computing.
QI2S represents a major step beyond ESA’s roadmap for the development of future payload processors.
The QI2S project reached a Technology Readiness Level (TRL) of 4 - technology validated in lab.
Project Context and Objectives:
Next generation satellites need intensive (near) real-time on-board processing capabilities in order to fulfil demands of compute intensive image and signal processing workload. Earth Observation Satellites (EOS) for hyperspectral imaging employ payloads, which typically sense radiation at wavelengths of 0.2 µm to 2.5 µm (UV-VNIR-SWIR), 3 µm to 5 µm (MWIR) and/or 8 µm to 14 µm (LWIR or TIR) and scan in up to thousands of pixels within one spatial dimension at once. Furthermore, per spatial pixel hundreds of narrow bands (typically 5-50 nm wide) collect highly precise information about the radiance spectrum of each imaged pixel (typically sampling 100-900m2 on Earth). This enables image-based detection of various materials and objects through the identification of their unique spectral radiance signatures. Such a strong concurrent differentiation regarding spatial and spectral information creates large signal data streams (e.g. the ASI-ISA agencies SHALOM hyperspectral project aims at imaging 200,000km2 per orbitday). The processing of such large data streams typically includes (representative for hyperspectral image processing algorithms, compliant with demands from European space agencies): radiometric correction for sensor-specific errors and radiometric calibration for upwelling radiance signals; cloud screening and atmospheric filtering, which computes the spectral reflection or emission profiles based on upwelling radiance and local features of the atmosphere; and automated information extraction algorithms, for specific material or object identification (e.g. anomaly detection, match filters, etc.). The resulting thematic maps and mosaics with different layers of extracted information can be rectified and geo-referenced to create a final information product for the end-user.
However, the potential of spaceborne hyperspectral imaging is far from being fully realized. One of the main bottlenecks is the long delay in response time, typically measured in weeks, caused by extremely large data volumes that need first to be transmitted to ground, then mobilized to the mission’s data exploitation centre, where they are prioritized, then submitted to effortful processing and only then distributed as useful (usually basic) products to the end-users. For example, a satellite orbiting at 680 km above earth having a ground speed of ca. 6,800 m/s, typically acquires imagery at rates of more than 2 Gbit/s, resulting e.g. in 20 Gbit of data for only one 70 km long ground image. Thus many terabits of data need to be downloaded at Gbit/s rates to facilitate near real-time hyperspectral data exploitation. Another reason why spaceborne hyperspectral imaging is far from being realized is of economical nature. Nowadays, most satellite compute systems rely on the use of hardware accelerators (i.e. FPGAs or ASICs) serving mostly a single purpose such as image compression. Even though FPGA accelerators are reconfigurable to some extent, it is very complicated and thus costly compared to designing or updating a software algorithm for an application specific or general-purpose processor (between which the developed RC64 and the QI2S prototype resides).
Most existing processors for space applications, such as Atmel AT697, Aeroflex UT699, Aeroflex Gaisler GR712RC and BAE Systems RAD750, provide performance levels below 1,000 MIPS, and are thus unsuitable for executing high-performance “next generation digital signal processing” (NGDSP) tasks in space missions. While NGDSP requirements are listed at 1,000 MIPS/MFLOPS, a more practical goal is 10,000 MIPS. Even the fastest, currently available space processor, SpaceMicro Proton200K, achieves only about 4,000 MIPS/900MFLOPS.
Recently, the US government has adopted Tilera’s Tile processor for use in space, in the framework of the OPERA program and the Maestro ASIC. Integrating 49 triple issue cores operating at 310 MHz, it is expected to deliver peak performance of 45,000 MIPS. Software development experience for the Maestro chip has encountered difficulties in parallelizing applications to the mere 49 cores of the Maestro. Some of the developments have underestimated the inter-core communication latencies involved in the tiled architecture of Maestro. Due to such difficulties, programmers are forced to cram multiple different applications into the many-core, resulting in additional difficulties regarding protection of each application from the other ones. Performance comparison for leading space processors versus year of introduction is plotted in Figure 1.
In order to fulfil the demand for satellite on-board compute intensive image and signal processing, the QI2S project aimed at implementing an innovative rad-hard, massively-parallel many-core computing system, augmented by specialized parallel processing software to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions.
Addressing the challenge embodied by on-board near real-time processing of HS data also opens the opportunity to provide similar technology for a wide variety of high performance space borne computing.
QI2S represents a major step beyond ESA’s roadmap for the development of future payload processors. Well-known ESA benchmarks consist mostly of basic signal processing algorithms like Fast Fourier Transform (FFT) and Finite Impulse Response (FIR) filtering and depend on the availability of flexible and scalable hardware and software solutions, since applications most likely change over space mission time and therefore space systems need to adapt within limited time frames.
Figure 1: Performance Comparison of RC64 Technology to Other Advanced Space Qualified Processors
2.2 Main Objectives
The principal objective of the Quick Image Interpretation System (QI2S) project is to initially demonstrate the realisation of the future required level of satellite computation power for different on-board laborious data processing tasks. QI2S does so by innovating, developing and validating specialised technology for fast on-board Hyperspectral Imagery processing. The development of a QI2S demonstrator is part of the effort to validate the usability and functionality of such system.
Thus the QI2S prototype concept implements an innovative rad-hard, massively-parallel many-core computing system and is augmented by specialized parallel processing software in order to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions.
The principal objective is further decomposed into the following QI2S technical development objectives:
OB1: Design, Develop and Validate QI2S many-core architecture
The foundation technology is Ramon Chips innovative RC64 many-core architecture – a custom high-performance 150 GOPS / 40 GFLOPS space qualified multiple-core parallel computing engine;
The QI2S many-core architecture is developed based on two unique architectural many-core design elements of Ramon Chips innovative RC64 many-core architecture –the Synchroniser/Scheduler and the shared memory system.
OB2: Develop QI2S technologies
QI2S leverages the QI2S Many-core architecture to research and develop the algorithms, the s/w building blocks, system software and the hardware platform. In particular the project developed, in addition to the many-core platform: QSBB – an application software implementing an interpreter with a finite number of specialized Building Blocks for HS data on-board processing and interpretation and: QMDL – a mission definition/command language, that will enable fast-turnaround reconfiguration of the processing procedure.
OB3: Integrate QI2S prototype
The QI2S project integrated a full QI2S prototype with the results obtained by achieving the previous objectives, namely the QI2S Many-core computing platform, the QI2S Software Building Blocks, the QI2S Mission Definition Language (QMDL), system software and related h/w integrated in a prototype with imagery and mission data interface to payload like links for imagery emulation, management & control.
OB4: Validate QI2S through ground-trials and simulation
The QI2S prototype was validated using a PC based imagery emulator & controller. Whose key The QI2S project validated the integrated prototype as well as the underlying architecture, components, QSBB and QMDL through a range of ground-trials. The QI2S validation process was performed at several levels, on the basis of real application scenarios, correlated to the development plan.
The QI2S consortium brings together 6 partners from 4 countries. The most part of the partners has successful past experience of mutual collaboration projects. Each partner has its own unique expertise in the individual domain which they have been assigned to for this project. The partners have built a strong and balanced consortium, including academic partners, research centres, large industries and SMEs.
Participant no. Participant organisation name Participant short name Country
1 [Co-ordinator] Elbit Systems Electro-optics ELOP Ltd. ELOP Israel
2 Digital Signalprocessing & Information Technology DSI GmbH DSI Germany
3 Compagnia Generale per lo Spazio CGS Italy
4 Ramon Chips RC Israel
5 ARTTIC SAS ART France
6 Technical University Braunschweig TUBS Germany
2.4 Project Structure
The work plan for the QI2S design, development, integration, test & validation project was divided into 5 technical Work Packages (WP). The WPs were further decomposed into specific tasks. For each task the leader and the contributors from among the consortium members were defined. A leader has been also nominated at WP level. Deliverables were defined for each work package.
Project Duration: 33 Months
Figure 2: Dependencies between Workpackages
Project Results:
3 Main S&T Results
3.1 Overview
The QI2S project has succeeded in proving the technological concept that combines a massively-parallel, many-core compute-hardware architecture, and an enabling parallel-computing, image processing software, leveraging a custom programming model, to demonstrate the potential to spaceborne on-board (near) real-time computing at performance-to-electric power ratio of approximately 15 GOPS/Watt or 2 GFLOPS/Watt.
The project developed detailed specifications, implemented them through prototype hardware and software, demonstrated the solution via end-user led ground trials and drove the overall approach to pre-normative industry acceptance.
The QI2S prototype, that was designed, developed and validated, features multiple chip components of the RC64 chip, embedded in a demonstration platform that includes a RC64 simulator, high-performance FPGA and parallel-processing application software developed to perform hyperspectral image processing algorithms.
Therefore, the QI2S prototype platform enables for the first time, to assess performance of challenging parallel-processing software on simulated many-core hardware architecture. Hence, it is an important “road sign” that represents a measure to the potential performance of operational, full scale, RC64-based as well as other parallel-processing, many-core architectures for future spaceborne systems.
3.2 The RC64 innovative architecture
The RC64 innovative chip architecture which is the basis for the QI2S development is briefly described herein below:
The RC64, is a novel rad-hard 64-core signal processing chip, which targets DSP performance of 75 GMACs (16bit), 150 GOPS and 20 single precision GFLOPS while dissipating less than 10 Watts. RC64 integrates advanced DSP cores with a multi-bank shared memory and a hardware scheduler. The programming model employs sequential fine-grain tasks and a separate task map to define task dependencies.
RC64 is currently being implemented in the framework of another project, as a 300 MHz integrated circuit on a 65nm CMOS technology, assembled in hermetically sealed ceramic CCGA624 package and qualified to the highest space standards. This new package and new process technology enable the inclusion of twelve integrated full duplex high speed serial links (HSSL) using CML SERDES interfaces on chip at 2.5Gbps rate each, with an aggregate 60Gbps throughput. Several protocols such as SpaceFibre are considered for HSSL, aiming at efficient connectivity among multiple RC64 chips and other FPGAs, ASICs. Also, faster and denser DDR3 SDRAM interface is made possible. Reed-Solomon ECC is employed to protect from DDR2/3 SEFI and SEE. The 32-bit wide DDR2/3 interface supports up to 25Gbps throughput. Other I/O interfaces in RC64 include SpaceWire for control, Parallel LVDS interfaces for advanced ADC and DAC devices connectivity and interface to flash memory.
The on-chip shared memory system of RC64 is based on each core having its own write-through data cache, an instruction cache, and a private store, supporting the unique task-oriented programming (TOP) model. All cores access the single shared memory with 256 ports and a 64-to-256 ports multistage interconnection network, enabling simultaneous access of all processors to shared memory with very little conflicts.
The on-chip 4MByte shared memory acts as a local-store memory. Access to off-chip DDR2/3 memory is facilitated by software-controlled DMA. This approach simplifies software development and it is found to be very useful for DSP applications, which favour streaming over cache-based access to memory.
The RC64 Core firmware architecture is depicted in Figure 2 and the concept of operation of the Many-Core is shown in Figure 3. A central scheduler assigns tasks to processors. Each processor executes its task from its cache storage, accessing shared memory only when needed. When task execution is done, the processor notifies the scheduler, which can subsequently assign a new task to that processor. Access to off-chip streaming channels, DDR2/3 memory, and other interfaces happens only via programmable DMA channels. This architecture and concept of operation have been thoroughly simulated and realized to the extent detailed herein below by the QI2S prototype. When a single RC64 has insufficient processing power for the application, multiple RC64 chips can be accommodated, as shown in Figure 4. The four chips are interconnected with high-speed channels. RC64 has been designed for integration with tens or hundreds of other RC64 chips, enabling very powerful digital signal processing in space.
The future, full-scale RC64 chip-based parallel-computing platforms will be customized platforms that will enable scalable performance using multiple chips communicating and processing concurrently, which can reach 10's of GFlops and 100's of GOPS.
Figure 3: The RC64 Core architecture
Figure 4: RC64 Many-core concept of operation
Figure 5: Single board DSP using 4 interconnected RC64 chips
3.3 The QI2S prototype implementation
The foundation technology is the RC64 Many-core – a custom high-performance 150 GOPS / 20 GFLOPS space qualified many-core parallel computing engine; QSBB – a software implementing an interpreter with a finite number of specialized building blocks for HS data processing and interpretation; QMDL – a mission definition/command language interpreter, that enables fast-turnaround reconfiguration devoid of elaborate time consuming testing attributed today to spaceborne payloads operational s/w change procedures.
Thus the QI2S prototype concept implements an innovative rad-hard, massively-parallel many-core computing system and is augmented by specialized parallel processing software in order to demonstrate the potential for on-board, (near) real-time, lightweight and low-power hyperspectral image processing system for spaceborne remote sensing missions. The development of a QI2S demonstrator is part of the effort to validate the usability and functionality of such system.
However, since the full scale RC64 chip is still under development in the work frames of other projects, the QI2S technology is implemented in this project with a downscaled FPGA board. The Image processing System (IPS) of the QI2S is integrated in the ASIC prototyping engine DNV7F1A from DiniGroup. It contains a high speed FPGA and as a stand-alone solution it can be integrated in a special housing containing power supply, fan, connectors, etc. The offered interfaces include among others Ethernet connectors and SFP+ interfaces for high data rates. A DDR3 UDIMM socket enables up to16GB of standard, off-the-shelf memory to be used by the FPGA. The final configuration of the DNV7F1A board used in this project contains:
▪ Xilinx Virtex 7 FPGA (7VX690T-1) with speed grade 1.
▪ 8GB DDR3 memory
▪ Chassis including power supply
The block diagram of the IPS is shown in Figure 5.
Figure 6: Block Diagram of the Image Processing System (IPS)
In the IPS, a hardware (task) scheduler dynamically allocates, schedules, and synchronizes tasks among the parallel processing cores according to the program flow. Hence, it reduces the need for an operating system (OS) and eliminates large software management/execution overhead. No OS is deployed to the cores. The complementary task oriented programming model (TOP) reduces efforts on parallel programming and is based on the C programming language, adding some few extensions. One of these extensions comprises of a so-called “Task Map” that explicitly embodies the program flow (i.e. task-dependency graph) and enables efficient scheduling of software tasks using the hardware scheduler, thus preventing software from relying on shared memory synchronization and eliminating the need to manually schedule tasks to processors.
The many-core computing platform is integrated with a software interpreter with a finite number of finely designed software building blocks for fundamental processing and interpretation of hyperspectral imagery.
The QI2S Software Building Blocks (QSBB) represents the algorithms that are used for image interpretation. There are three QSBBs arranged in a consecutive order (Figure 7). The intermediate result can also be accessed for reading to offer debugging capabilities.
These building blocks are designed to form the fundamental toolbox comprising essential mathematical operations and filters which accommodate radiometric correction, atmospheric correction and materials/objects detection by identifying characteristic spectral signature features within a certain spectral range and threshold:
▪ Radiometric Calibration
The raw image from the hyperspectral camera is processed for authentic radiometric correction (i.e. for a real imager), which is separated into six sub blocks. This stage is required to correct the uniformity between the micro sensors for each spectral band and each pixel on the vertical row of sensors. In addition this algorithm handles the rotation registration and binning issue to create the exact number of bands required to the analysis of the spectral domain.
▪ Atmospheric Correction
This stage with five sub-blocks is required to correct the atmospheric effects that were added to the sensed image on the optic flow between earth and satellite. In addition, this algorithm handles the transformation of images from radiance units to those of reflectance. The algorithm uses two passes to reduce the two degrees of unknown water vapour, and the visibility range.
▪ Material Detection
The last stage in the image interpretation is the material detection, which consists of four sub-blocks. The output of the algorithm is a grey scale level for every pixel representing the probability of the material presence. A threshold transforms the grey scale result in a binary result. The three materials that the implemented algorithms detect are: green vegetation, clay minerals and plastic greenhouse. Cloudy pixels or no detection is indicated as well. In hyperspectral imagery, different processing procedures that usually make use of similar mathematical functions but use different parameters, variables, thresholds and apriori data, need to be applied for extraction of different types of information from the same data set.
Figure 7: Data Flow of the Image Processing System
A PC based system named Emulator & Controller (EC) represents the EO payload interface, emulating hyperspectral Imagery and performing management, control and monitoring of the IPS. Aside the IPS, the EC is also a part of the QI2S prototype. The main features of the EC are:
• Emulate: Row-wise streaming of hyperspectral images via a high speed link to the IPS.
• Control:
▪ Mode management and application control
▪ Work flow and operation configuration via the QMDL software
▪ Transferring of pre-calculated mission support data (tables) to the IPS, Start and stop IPS processing
• Analyse:
▪ Reading and analysing the correctness of the data and presents the relevant timing measurements
▪ Request and interpret BIT of the IPS, verify intermediate results and record all kinds of inputs and outputs
The EC is equipped with a high-speed serial card (Xilinx KC705) to emulate the payload with a realistic data rate of 2 Gb/sec (Figure 8). To enable the controlling, emulating and analysing capabilities of the EC the test software package GSEOS V is used. GSEOS V is designed to support all stages of experiment development, from bench checking and spacecraft integration up to “quick-look” during flight operation. It supports testing of units under near real-time conditions by using a data-driven concept in contrast to less efficient¬ polling. It provides response times of less than 10 μs. It is user configurable and therefore very flexible. GSEOS V is able to:
• Interpret the data to be transferred from EC to IPS (e.g. digest the QMDL Config Packet or Image File)
• Send those data to the IPS.
• Control the IPS to enable several work flows (normal processing / verification).
• Read, analyse and display the outputs of the IPS.
• Store all the inputs and outputs for future reference.
Figure 8: QI2S Demonstrator (block diagram)
These procedures must be tailored by simple mission commands and therefore a flexible Mission Definition Language (QMDL) is also developed, based on the GSEOS V software package on the EC side and on interpreting s/w on the IPS side, to enable fast-turnaround reconfiguration of the QI2S hyperspectral imaging interpretation process, setting different processing chains for different hyperspectral detection scenarios.
3.4 Performance Assessment for a RC64 Full-scale Chip
The developed QI2S application software runs on three platforms:
1. Software emulator
2. Software simulator
3. FPGA platform
However, none of these platforms achieves a cycle-accurate representation of executing QI2S application on the target, full-scale architecture of the RC64, as follows:
1. The software emulator executes the code on Intel X86 architecture. Tasks are executed sequentially. I/O is only roughly estimated. Memory constraints are ignored.
2. The software simulator employs Tensilica DSP cores that are differ from RC64 DSP cores produced by CEVA.
3. The FPGA platform emulates only 16 Tensilica RISC cores executing at 35MHz, performing 560 MFLOPS and provides only a partial simulation of the rest of the RC64 components and I/O. The full-scale RC64 operates 64 VLIW CEVA cores executing at 300MHz, performing up to 38GFLOPS.
The performance estimation for a full scale RC64 platform is based on analysis of the computational bottlenecks of QI2S. This application consists of floating point operations. Assuming complete parallelization, such that, each computational step is parallelized in a balanced manner to all 64 cores, we note that higher performance is possible if some or all computations are converted to fixed-point arithmetic.
Measurements of floating point computations counted on the emulator shows that 1000 image “rows” (1 “row” = 1000 spatial x 320 spectral pixels) require some 360M floating point operations. 93-94% of all floating point computations are spent for interpolation methods. The measurement reliably represents actual computations, regardless of the architecture on which QI2S is executed. The measurement assumes that there are no I/O bottlenecks and no memory congestion. This is a reliable assumption for a well-optimized application.
RC64 is designed for a performance of 38 GFLOPS (Giga floating point operations per second). Thus, the RC64 estimated performance based on the QI2S results is:
This is an upper bound on performance of a single RC64 chip when all arithmetic employs floating point operations. Expected actual performance in floating point is somewhat lower because:
1. Non-arithmetic parts of the code, including iterations and other instructions, are not counted. Typically, these are optimized so that they do not affect performance too much. However, this analysis has not been performed.
2. The reporting counting has not been validated by a third party. It is possible that correcting errors in measurement may reduce estimated performance by a small margin.
When pixel rates higher than 34 Mpixels/second are required, multiple RC64 chips may be employed. The image data can be divided into several RC64 chips operating in parallel. For instance, consider a hyperspectral imaging LEO satellite where each pixel’s GSD (ground sample distance) is 10×10 meter (e.g. the hyperspectral payload designed for the ASI-ISA agencies SHALOM mission), the imager swath spans 1000 pixels, and the satellite ground velocity is approx. 7,000 meters/second. The resulting synchronous scanning pixel rate is 224 Mpixels/second, and 7 units of RC64 are required to handle this data rate. A more realistic asynchronous pixel rate of 75Mpixel/seconds (due to both FPAs readout limitations and even more important, due to “back-slew” motion compensation maneuver required to increase signal integration time) would require only 3 units of RC64 for on-board, real-time processing of hyperspectral imagery!
Converting floating point computations into fixed point should result in even faster processing rate. Indeed, considering that the pixel image data is typically digitized into 12 to 16 bits, 16-bit arithmetic may suffice for most computations. The fixed point peak performance of RC64 is 150 GOPS, 4 times faster than FLOPS rate. Thus, the estimated data rate of a single RC64 in fixed-point computation is 135 Mpixels/second. This one-RC64 data rate is more than sufficient for the hyperspectral camera imagery stream example mentioned above.
Potential Impact:
4 The potential impact
4.1 Socio-economic impact and the wider societal implications
Building upon the innovative technologies described above, the QI2S prototype development results, initially demonstrate and pave the way to an operational QI2S that will enable significant reductions of delay in the delivery of Earth Observation Hyperspectral processed/interpreted data to the end user, from days or weeks to real-time or near-real-time by designing, developing, integrating and validating a full scale platform.
The need for spaceborne hyperspectral remote sensing emerges dramatically and is driven by a continuously increasing range of important applications in various fields. Many evolving applications require fast imagery data exploitation to support real-time decision-making. For example the cases of monitoring evolving incidents and disaster analysis: e.g. for fighting wild fires (real-time information on expected spreading directions based on forest biomass analysis and humidity conditions is highly desired), or furthermore for search & rescue in case of accidents in remote or maritime locations and scenarios. Such rescue operations demand immediate detection of concealed or drifting debris, survivors or bodies. Additionally, water contamination, air pollution, atmosphere conditions monitoring and precise agriculture (disease/stress precise detection, customized fertilization etc.) are also some of the many fields that can significantly benefit from real/near real-time spaceborne hyperspectral imagery data exploitation which is accessible and affordable.
QI2S helps usher in the next generation of lightweight HS satellites able to provide a complete suite of on-board imagery processing and information extraction by automated algorithms, for day-to-day operations, such as oceanographic mapping, as well as for evolving incidents, such as post disaster analysis.
QI2S responds to the need for EU to solve Societal Challenges and to the need for space companies to remain competitive by creating and selling successful services and space infrastructure that contribute in solving these challenges. Hyperspectral missions and QI2S can provide precious information to the society allowing Earth monitoring in real time. QI2S on board of new space missions will allow increasing our knowledge related to our Earth, helping us to better understand ourselves and our environment.
Spaceborne on-board (near) real-time imagery processing opens not only the opportunity to provide HS “avant-garde” services from space but also provides for similar technology to serve a variety of other high performance spaceborne remote sensing computing applications such as SAR, on-line image data compression, real-time EO object detection & recognition, real-time image rectification and geo-location, etc. Hence, it will open the space sector to a wider range of new applications and also to a new way of delivering space services matching evolving application requirements to broader public communities and potentially to new user markets.
4.1.1 Competitiveness
The product
Following this vision, two main assets have been individuated for QI2S market opportunities:
1. Onboard Processing Platform Exploitation: this technology will be able, in the future, to reach performances of hundreds of GOPS and tens of GFLOPS. The development of this disruptive, key-enabling technology will radically innovate the on-board processing units, compared to existing on board hardware solutions, and will increase on-board compute performance capabilities by 10-100x. The hardware platform is demonstrated for real-time on-board analysis of hyperspectral data, but can be used on a wide range of different processing performed directly in space. Moreover such design can simplify the instrument design, itself leading to reductions in mass, power and volume for missions as remote sensing where multiple instrument components necessitate optimization of these resources. In this way, strategic application of information technology advances can lead to measurable improvements in both instrument data reduction and design. From this point of view QI2S product will substantially help accelerate and usher in the next generation of lightweight, low-cost Hyperspectral EO satellites.
2. EO Products Fast Delivery Service Exploitation: The need for innovative EO services based on spaceborne hyperspectral remote sensing services emerges dramatically and is driven by a continuously increasing range of important applications in various fields. Many EO services require fast imagery data exploitation to support real-time decision-making or to provide the citizens with up-to-date information for our daily life. Our vision is to realize future earth observation missions capable of delivery EO products directly to the user at each satellite pass. The reception of the satellite products has not to be so heavy. The data processing on board reduce the amount of data need to be downloaded and facilitate real-time hyperspectral data exploitation.
The QI2S exploitation is unique in the field of future EO mission because it blends fully the hardware development for disruptive on board processing platform and the on-board image processing, which support all key aspects of real time EO service delivery. QI2S has been tested in laboratory (TRL 4) through the WP 5 activity. This technology will be able, in the future, of reaching very high performances, although for demonstrational purposes it is now implemented on a downscaled environment.
The Market
A large number of EO satellites are expected to be launched in the next years. As reported by Euroconsult in “Satellite-Based Earth Observation” Market prospect to 2022, about 180 new EO satellites will be launched over 2013 to 2022. Assuming that each one needs a framework for on-board downstream applications, then the global market is 180 units per year on average over the next 10 years. Supposing a cost of €5M per system, this could represent a global market of € 900M in 10 years.
Growth expected in the market of space payload systems as QI2S is based on two processes:
• On-board Processing Platform: enabling novel missions and applications that were thus far impossible in space. The growth rate of adoption of this new technology is expected to be very fast, much faster than the base growth in the space industry.
• EO Products in real time: the transition of computations from ground stations to space, in order to accelerate time-to-results by eliminating the need to transmit high data volumes from space to ground;
Target Stakeholders
The target stakeholders for QI2S range from important Space enterprises, SMEs and institutions at European and international level. The principal entities, which can be interested in the outcomes of QI2S project are:
• Governmental institutions,
• Space Agencies,
• Large and medium companies active in the Aerospace domain
• Large and medium companies active in the EO services domain
About EO services each stakeholder can have different needs related to the image characteristics. The mission design and the sensor definition will have to take into account this variability providing a camera with acquisition capabilities able to meet the full spectrum of them, both civil and government entities requirements. As a possible set of user applications it is worth to mention:
• Worldwide complex emergencies for situation awareness and rapid damage assessment;
• Natural disasters (floods, earthquakes, volcanic activity, tsunami, drought, vegetation fires, etc.);
• Homeland security and intelligence applications: surveillance and reconnaissance;
• Technology risks, major transport routes, accidents associated with transport, industrial activities.
About the Large and medium companies active in the Aerospace domain they have a great interest on the Onboard Processing Platform asset, while the Large and medium companies active in the EO services domain are concentrated on the EO Products Fast Delivery Service.
Potential Competition
The future exploitation of QI2S leans on two assets: EO Products Fast Delivery Service asset and On-board Processing Platform asset. It has to be taken into account that the EO Products Fast Delivery Service is a new technological frontier that goes beyond competition. Before having a real competition in this asset, the On-board Processing Platform has to be quite ready to be launched.
QI2S has developed a very innovative idea, where a first approach to Hyperspectral Products Fast Delivery Service has been developed at prototype level. This approach will allow the QI2S team to be ready to deliver new added value products, processed directly on board, once the processing platform will be ready for space. Currently the completion for such system can be analysed focusing on the On-board Processing Platform asset, where the industrial and scientific community is concentrated on.
While the commercial processors have short life and are quickly replaced by new generation architectures, in the space sector processor performance research goes hand in hand with reliability. Moreover in this sector, is usual to have long-time programs that require solutions supported in the long term (up to 15 years). This entails, of course, a chronic delay of the space sector compared with the SOTA technology on the commercial and industrial market.
The computing demands and the low power requirements of current sophisticated space missions are starting to exceed the performance capabilities delivered by state-of-the-art flight qualified processors, so the high performance of the future RC64 multi-core processor, becomes an innovative strength in QI2S project. In this project the research activities have been carried on and the processor has been adapted for Image Processing System.
An overview of the competition in space processors (current and future processors) is given in Table 1. All the listed processors are or will be RAD-HARD.
Processor Manufact. year Freq,
MHZ MIPS MOPS MFLOPS Power cons. [watt] Watt/
MFLOPS
RC64 Ramon Chips 2016 300 150,000 38000 8 2.1 x 10-4
Proton200k Space Micro, Inc. 2008 4000 900 4,24 46 x 10-3
Rad5545 BAE 5200 3700
RAD750 BAE 2001 200 266 5
NGMP-LEON4 Aeroflex-Gaisler Future (2020-2025) 150 75 x 10-3
GR712RC-LEON3 Aeroflex-Gaisler Future- later 2015 100 200 DMIPS
[140 DMIPS/core]* 200 MFLOPS 1,5
UT700-LEON3 Aeroflex-Gaisler 2014 166 1.2 DMIPS / MHz [233 DMIPS/core]* Max 4
UT699-LEON3 Aeroflex-Gaisler 2011 60 75 DMIPS [92 DMIPS/core]* 5.5 4 x 10-2
AT697F-LEON2 ATMEL 2009 100 86 DMIPS 23 MFLOPS 1
AT6981-CASTOR Future- later 2015 200 150 MIPS 40 MFLOPS 36 x 10-3
OPERA MAESTRO
(ip removed) Tilera-EZ Chips Future- later 2015 350 5000 MFLOPS 18 W 52 x 10-3
*With latest compiler, 0 wait states on SRAM and cache enabled
Table 1: Space Processors Performance Comparison
In the Europe, the main manufacturer of processors to space applications is the Swedish Gaisler Research (acquired in 2008 by Aeroflex Inc), a designer of rad-hard IP (semiconductor intellectual property core) for space applications, including the open source LEON processor.
The new company, Cobham Gaisler AB, (Aeroflex-Gaisler) provides IP cores and supporting development tools for embedded processors based on the SPARC architecture and it is specialized in digital hardware design for both commercial and aerospace applications.
Several versions of the LEON processor have been developed by Aeroflex Gaisler:
• LEON2 based processors
• LEON3 based processors
• LEON3 FT- A fault-tolerant version of LEON3
LEON3 FT - a fault-tolerant version of the standard LEON3 SPARC V8 Processor
Aeroflex Gaisler provides 3 different microprocessors LEON3 FT based:
• UT699-LEON3 is a pipelined monolithic, high-performance, fault - tolerant SPARC TM V8/LEON 3FT Processor. The UT699 provides a 32-bit master/target PCI interface, including a 16 bit user I/O interface for off-chip peripherals. The clock frequency is 60MHz and achievefrequency is 60MHz and achieves the performances of 92 DMIPS/core. Its Power Consumption is 5,5 W.
• UT700-LEON3 features a seven stage pipelined monolithic, high-performance, fault-tolerant SPARC V8/LEON 3FT Processor. The clock frequency is 166MHz and integer performance is 1.2 DMIPS / MHz. Its Power Consumption is 1,5 W.
• GR12RC-LEON3 a dual-core LEON3FT 32-bit processor, with two fully SPARC V8 compliant integer units and two high performance fully pipelined IEEE-754 floating point units. The clock frequency is 100MHz and achievefrequency is 100MHz and achieves the performances of 200 DMIPS and 200MFLOPS. Its Power Consumption is 1,5 W.
PROCESSORS WITH PERFORMANCE COMPARABLE TO LEON3
BAE System Electronic Inc. realized the RAD750 that was released in 2001, with the first units launched into space in 2005. The CPU has 10.4 million transistors, nearly an order of magnitude more than the RAD6000 (which had 1.1 million). It is manufactured using either 250 or 150 nm photolithography and has a die area of 130 mm2. It has a core clock of 110 to 200 MHz and can process at 266 MIPS or more. The CPU can include an extended L2 cache to improve performance.
The CPU itself can withstand 200,000 to 1,000,000 rads (2,000 to 10,000 gray), temperature ranges between –55 °C and 125 °C and requires 5 watts of power. The standard RAD750 single-board system (CPU and motherboard) can withstand 100,000 rads (1,000 gray), temperature ranges between –55 °C and 70 °C and requires 10 watts of power.
Another similar product is Proton 200k manufactured by Space Micro Inc. (USA). It is a high-speed, SEE-hardened DSP available for space and satellite applications available from 2008.
The main manufacturers involved in electrical component to space applications are developing a new generation of processors, available in the next future. The new LEON4 core is an evolution from the LEON3 core with improved performance thanks to wider internal buses, modified pipeline and support for a Level-2 cache. A LEON4 evaluation board with a dual-core processor system is currently available.
The GR-CPCI-LEON4-N2X evaluation board has been designed for evaluation of the Aeroflex Gaisler LEON4 Next Generation Microprocessor (NGMP) functional prototype device. The NGMP functional prototype is a system-on-chip with four 32-bit LEON4 SPARC V8 processor cores connected to a shared 256 KiB Level-2 cache and several high-speed interfaces, including an 8-port SpaceWire router and dual gigabit Ethernet interfaces. The architecture provides improved support for debugging and software partitioning together with extended support for both symmetric and asymmetric multiprocessing.
The Atmel AT6981 Castor device is a LEON2-FT based system-on-chip with multiple integrated peripherals including an eight-port SpaceWire router and three internal SpaceWire engines each containing three DMA channels, an RMAP initiator, and an RMAP target.
The RAD5500 family of radiation-hardened processors use the QorIQ Power Architecture with processor cores based on versions of the Freescale Technologies e5500 core. The RAD5510, RAD5545, and RADSPEED-HB (Host Bridge) are three systems on a chip processors implemented with RAD5500 cores produced with 45 nm SOI technologies from the IBM Trusted Foundry.
The RAD5545 processor employs four RAD5500 cores, achieving performance characteristics of up to 5200 MIPS and over 3700 MFLOPS.
Based on the RAD5545, the RADSPEED-HB is intended for host processing and data management support for one to four RADSPEED DSPs. The RADSPEED-HB replaces a secondary DDR2/DDR3 memory interface connection found on the RAD5545 with connections for RADSPEED DSPs instead. (Note that RADSPEED DSPs are entirely different processors that are specialized for digital signal processing and are not to be confused with the RADSPEED-HB, which serves as a host bridge.)
The Maestro processor was designed based on the TILE64 processor. It has 49 cores arranged in 7 by 7 meshes on a chip. It also added an FPU in each core for high performance for floating point operations. Since the Maestro processor was designed for space applications, the processor has been radiation-hardened by design, which resulted in reducing the number of cores from 64 to 49. The Maestro processor runs at up to 350 MHz clock frequencies. On the Maestro processor, we implemented an FFT and an image processing application called CRBLASTER and evaluated the performance. The FFT is a well-known and commonly used signal processing kernel. CRBLASTER is a parallel-processing image-analysis application that does cosmic-ray rejection on CCD (charge-coupled device) images using the embarrassingly-parallel L. A. COSMIC algorithm. Both applications were written in C. CRBLASTER uses the high-performance computing industry standard Message Passing Interface (MPI) library. The achieved performance of the FFT was up to 3,813 MFLOPS, and the speedup compared to single tile was 46.4 using 49 tiles. The speedup for CRBLASTER, which was memory-bound, was up to 12.5 using 36 tiles.
Potential Risks
The main identified risk for implementing our QI2S individual exploitation plan lies in the low level of maturity of the system. The TRL achieved at the end of the project is 4 (validated in laboratory). This level of technology readiness is far from to be ready to use QI2S in future space missions, if the research on this technology will not be continued to reach a higher level of maturity.
4.1.2 QI2S dissemination activities
QI2S initially envisaged as part of Work Package 6, T6.1 “Dissemination activities” to establish at project start a detailed dissemination plan (including communication strategy and contributions). Although defined and coordinated by the leader of T6.1 partner ARTTIC, this plan was to be agreed upon by all project partners, which would have to implement its actions along the project duration. This plan aimed at defining the target audiences, objectives, communication channels, tools and support material, activities, responsibilities, timing, budget allocations. It also was to select dissemination actions to implement according to the plan.
The dissemination plan was established and agreed upon during the project first year. It focused on the following target groups and objectives:
Target Group Objectives
Primary target group - Industry Raise awareness to the technology developed by QI2S solution and the different possibilities it can create for onboard EO image interpretation systems
Secondary target group – Academy Promote research on hyperspectral imaging, promote the use of hyperspectral imaging in research in other topics such as environment studies, agriculture, space research
Tertiary target group - General Public Raise awareness in the public regarding hyperspectral imaging, and EO image interpretation.
The table below shows the overview of the scheduled actions in order to reach these targets:
Activity Project Month
Project presentation materials (Project logo, project PPT, document templates M3
Public website M4
Printed materials (Project Brochure, poster) M12-M24
Scientific Publications M1-M33
Participations in conferences and events M1-M33
Project dissemination event M33
The project communication materials
The QI2S Project dissemination strategy relied on diverse communication materials. It included the items listed below.
• The project logo was drawn at early stages of the project. A few options were presented to the consortium member who decided on the one which reproduced on every QI2S document, including this deliverable (at the first the page). The project identity was then built around this logo, its spirit and colours. It resulted in the production of the project presentation / communication templates to be used by all partners when communicating about QI2S.
• The project printed material covered a standard brochure and Poster. Their finalisation and printing were delayed due to the fact that the dissemination team waited for promising technical results to be included in their content. They were distributed to partners to support their speech at conferences at mid-project.
Figure 9: Project Logo
Figure 10: Project Brochure
Figure 11: project poster
QI2S publications and participation in conferences and events
During the course of the project and along project findings and promising results, partners made research developments and outcomes available to the scientific community through 2 scientific publications, mainly in peer-reviewed journals. On top of that partners has participated in several events and represented QI2S project. The list of publications and events are available in section 4.2.
QI2S dissemination event
The consortium planned a dissemination event hosted by CGS in order to present the project's results to stakeholders in the industry. Unfortunately not long before the event the QI2S system, still in a mature stage suffered from technical problems which did not allow the consortium to present it in public. Due to the summer vacation and the fact that the project has ended quite after it, the consortium was not able to plan another dissemination event and we were forced to let go of this activity in order to put our efforts on final R&D actions.
Collaboration with other FP7 projects
Due to the fact that many members of QI2S consortium are also partners in the ongoing project MacSpace (607212), a close communication and collaboration was formed between the two projects. MacSpace will continue after QI2S has finished and will potentially exploit knowledge created in QI2S project by including additional work on the hyperspectral application. Some dissemination activities were shared between the two projects since they involved and presented both projects, especially those made by RC who is the coordinator of MacSpace project.
Future dissemination activities done by MacSpace will also potentially include information about the QI2S project, and therefore will continue dissemination about the QI2S project.
List of Websites:
4.1.3 QI2S public Website
The concept for the implementation of the public website was defined as follows:
Target audience Large audience ranging from experts from industry and research, scientists community, users of the knowledge and technologies resulting from this research work, and stakeholders, policy makers, European Commission as well as the general public.
Communication channel The domain name : http://www.qi2s.eu/
Objective Support dissemination of information about QI2S.
Present the main benefits of QI2S regarding Earth Observation (EO) Hyperspectral (HS) technologies.
Expected impact Enhance the project visibility, create awareness of the major scientific and technological achievements and contribute to creating a positive view of the European but also world-wide community of equipment stakeholders to enable their adhesion and possible contribution to the QI2S developed standards.
Content Section Content
Home Page Project summary
News and Events The latest information about the project and events.
About the project State of the Art – Background, objectives, activities and expected results of QI2S project (Page will be updated along the project)
Partners Presentation of the consortium. Identification and short presentation of the partners as well as presentation of their role in the project, with the links to their own websites
Publishable Outputs Project publishable outputs identified under the following groups:
• Press Releases & Articles
• Downloads (Flyers for various events and dissemination activities)
• Publications (also includes attended events with presentations)
Page will be updated along the project
Links Links to related projects, FP7 sites, European sites
Contact us Contact details of the QI2S coordinator, and a contact form
Date(s) Preliminary version was defined M3. The web site was published internally in M4 for final review. Continuous updates are made regularly.
Leader CGS
Contribution • For the elaboration of the content, Project Office team first prepared drafts based on information available, and then contacted the partners for required contributions/modifications.
• Editing and quality check was organised by the PO team.
• Validation: Notification procedure organised by the PO team
