Skip to main content

Hyperspectral imaging technology for the quality inspection of fish products

Final Report Summary - SPECTRAFISH (Hyperspectral imaging technology for the quality inspection of fish products)

Executive Summary:
Quality assurance is one of the most important goals of any industry. The ability to manufacture high-quality and safe products consistently is the basis for success in the highly competitive food industry. In this context, the need of rapid food inspection methods is growing.

Today, quality and safety of fish are assured by multiple determinations of external and internal parameters. Such on- and off-line methods are invasive, time-consuming and sometimes inconsistent. The development of a rapid, objective, reliable and non-invasive method would represent a significant advance for the finfish industry and consumer trust on this sector.

A European consortium of 8 members counting small and medium enterprises of the fish industry, as well as research centres, have participated in the SPECTRAFISH project that aimed to develop a novel pre-competitive system for the inspection of fish products based on the use of hyperspectral imaging (HSI) technology. By integrating two conventional optical sensing technologies (i.e. computer vision and spectroscopy) into unique imaging sensors, HSI provides not only spatial information, but also information for each pixel in an image, therefore providing the answer to the question of “where is what”.

During the first year of the proposed project, a laboratory test-rig was used to train and validate numerous quantitative models for the measurement of quality and safety attributes of finfish fillets based on the most critical image features extracted from hyperspectral images of finfish fillets and their reference attribute values, which were measured by using traditional instruments and sensory analysis. During the second year, a reliable pre-competitive prototype of a HSI system was built and validated for the rapid, non-invasive and simultaneous measurement of several safety and quality attributes. Chemometric models were custom made for the prototype system during validation testing, and the most successful model, for fat in salmon, was used to successfully test the performance of the prototype in industrial conditions at a fish-processing plant.

Once SPECTRAFISH is ready for commercialization, could positively impact the competitiveness of the European fish sector through the automation of the inspection process and consequently saving up processing time, which could allow for more product throughput and increased volumes of fish produced, less spoilage and downgrading of fish during processing, and subsequent reduced risk of fish recontamination and potential shelf-life extension. These will be major drivers for stimulating market demand for the technology among fish processors.

Project Context and Objectives:
The overall objective of the SPECTRAFISH project was to develop a pre-competitive hyperspectral imaging (HSI) device for the automatic, rapid and non-invasive measurement of quality and safety attributes of finfish fillets based on the most critical image features extracted from hyperspectral images of finfish fillets and their reference attribute values.

In order to achieve the above, the technical and operational objectives that needed to be fulfilled are provided below.
1. To perform a technical review of the specific needs of the participating SMEs, to clearly define the finfish quality parameters to be measured by the hyperspectral imaging technique and to draw-up the system specifications aligned with SME requirements, regulatory and market considerations.
2. To set up a laboratory test-rig and to perform a series of laboratory trials to acquire high-resolution digital images of finfish fillets at different wavebands, and make the necessary adjustments and modification to ensure the system is robust for subsequent experiments.
3. To perform laboratory experiments with the hyperspectral imaging technique to identify the most critical image features extracted from near infrared hyperspectral images of finfish fillets and with the traditional methods to obtain the reference values of the attributes.
4. To establish chemometric relationships between the spectra and the selected quality and safety indicators.
5. To design and build the SPECTRAFISH hardware.
6. To develop the general SPECTRAFISH software.
7. To integrate the system hardware, software and User Interface in order to provide a pre-competitive SPECTRAFISH prototype that can be validated in industry.
8. To validate the SPECTRAFISH prototype against chemical, physical and microbiological parameters of selected species.
9. To install the SPECTRAFISH prototype fish processing lines and to run production trials to demonstrate its ability.
10. To make improvements to the prototype system based on feedback from the validation work, and to carefully outline post-project scaling-up guidelines and development work towards full production.
11. To facilitate the uptake of the SPECTRAFISH results by the participating SMEs as well as a wider audience by carrying out a comprehensive series of knowledge transfer and training activities.

Project Results:
With the aim to develop a novel system based on NIR hyperspectral imaging (NIR HSI) for quantitative and non-invasive measurement and visualization of quality and safety attributes of finfish fillets, the project started by reviewing the technical needs of the participating SMEs. Fish species of interest were selected: cod and salmon, as they are common and relatively high-value fish, and they have different attributes (e.g. salmon is fatty and cod is not; salmon is coloured and cod is white; cod commonly contains nematodes, but salmon less frequently). A preliminary list of attributes of interest was also agreed, based on end-users proposals. RTDs gained in-depth understanding of the technological needs of the finfish SMEs, as well as of common attitudes among fish processors and sellers regarding assessment of fish, by visiting several fish-processing sites in the town of Hirtshals (Denmark). A comprehensive literature review was carried out in order to select those parameters for which laboratory testing was well-established, which led to agreement on the final list of attributes to be studied in the project. A review of patents and legislation that had to be considered for the definition of the specifications of the system was carried out. As a result of the information gathered, the overall system specifications and performance characteristics of the SPECTRAFISH system were defined.

The technical work started by setting up a laboratory test-rig based on hyperspectral imaging (HSI) technology. The most appropriate illumination, spatial and spectral resolution, motor speed and frame rate settings for ideal image acquisition conditions were selected. Other parameters such as ROI images collection method, spectra pre-processing, pixel/spectra detection method and multivariate analysis method were optimised for the development of a protocol for further HSI analysis. Additionally, the consistency of the acquired images of fish fillets was proved through statistical tests. Following this, high-resolution digital images of cod and salmon fillets in the visible (400-950 nm) and near infrared (900-1700 nm) spectral ranges were acquired. In parallel to the acquisition of images, reference analyses of the selected attributes of interest were performed for each species using protocols and training provided by CSIC.

Results obtained showed that NIR spectra were found to have more information about parameters directly related to the freshness of fish in terms of safety features (WHC, K-index and TVC) so this range was selected as the best candidate for further analysis. Spectral data analysis and chemometric tools were used to correlate the spectra to quality and safety parameters, whereby several chemometric tools were tested in order to condense the information provided by the spectral responses of each sample image pixel. Partial Least Square Regression (PLS) was used to develop predictive equations for each attribute. Correlation coefficients of between 0.65 and 0.92 in cross validation were obtained. Selection of important variables was conducted to identify the most important wavelengths/variables that had the greatest influence on predictions throughout the whole wavelength range. The results obtained were used to establish a procedure protocol to be used with the forthcoming hyperspectral images acquired with the SPECTRAFISH prototype.

Based on the system specifications defined, as well as knowledge acquired through in previous work, hardware and software was designed for develop SPECTRAFISH which intend to be a novel system based in NIR hyperspectral imaging (NIR HSI) for quantitative and non-invasive measurement and visualization of quality and safety attributes of finfish fillets. The Quality Functional Deployment (QFD) methodology was used to efficiently manage the design, building and testing of the prototype. A function means analysis table was used to identify the most suitable components for each function of the prototype based on performance, ease of fabrication and cost. A block diagram was used to map connectivity between the parts of the different sub-systems. System specifications require that the prototype be suitable for food contact, so exterior surfaces are constructed from stainless steel 304, Teflon and glass, the design avoids protuberances and crevices where fish material can accumulate, and enclosures have ingress protection rating of IP66 to facilitate cleaning with a high pressure water jet. The prototype underwent an Operations Qualification testing process in IRIS to establish that all systems were functioning correctly, prior to delivery to CSIC for modelling validation. A further Process Qualification testing process was carried out at CSIC to confirm that the prototype was performing within system specifications.

The final prototype is comprised of a conveyor suitable for the food industry, and cabinets mounted on a frame above the conveyor, housing a focussed diffused halogen lamp, the spectral camera and a touchscreen computer. The spectral camera contains a spectrograph, a high speed camera and a fixed focal length camera lens. A hyperspectral image of 320 x300 spatial pixels and 256 spectral points per pixel is collected from a real-world area corresponding to 750mm along the length of the conveyor. Acquisition of the hyperspectral image is triggered automatically by camera recognition of a Teflon bar across the width of the conveyor, which indicates the beginning of the sampling zone.

An automated acquisition routine acquires bright and dark references, followed by the hypercube. The recently acquired hypercube is then analysed using PLS analysis for several quality parameters appropriate to the fish species being tested, and a logic table is used to apply a single quality index to the fillet based on the combined results of the quality parameters. The raw hypercube data as well as the analysis results are stored online at a Dropbox account, to prevent filling the hard-drive storage of the prototype computer, which would risk slowing the control, acquisition and analysis functions. The acquisition of the hypercube, the analysis of data, the data storage and the User Interface display are run in parallel using a multi-process architecture that makes use of the multicore division of the CPU. The User Interface provides flexible control of the acquisition through reconfigurable settings for the camera. The most recent results are displayed in series, and are accessible through a simple database, with 2D distribution maps of the quality index and of the individual quality parameters, with visual indicators of out of specification samples.

The installation of the prototype in the pilot plant facilities at IIM-CSIC was successfully carried out. However, models developed in WP2 were not transferable from UCD instrument to the prototype and so new models were built using samples analysed by the prototype before validation trials.

In particular, in cod, low correlation coefficients (<0,60) were obtained for the chemometric models that predict the three quality parameters, but calibration and cross validation errors were higher in TVB-N (total nitrogen volatile bases) comparing with TVCp (psychrophiles) and TVCm (mesophiles). However, blind validation results demonstrated SP-prototype could distinguish two different quality classes.
In salmon, industrial partners decided to focus the study in fat content as the most relevant parameter. Two different experimental designs for sample acquisition were checked and concluded that models improved when it is guaranteed the match between the spectral data and the analytical result in each sample (from r2=0,680 to r2=0,760). Different spectral pre-processing treatments were compared in the calibration and validation trials. Obtained model with the lower error of prediction (in terms of RMSE) was able to predict 22 out of 30 samples with <15% error and 26 out of 30 with <25%error. Thus, mapping of fat distribution was obtained, thus demonstrating the feasibility of using the pre-competitive hyperspectral imaging prototype to measure fat content in salmon fillets.

Finally, Industry trials were successfully carried out in a real industrial environment, where the prototype was used to measure the fat content of salmon fillets over continuous use. The prediction accuracy of the chemometric models was found to be 2.3% RMSE of fat content, which is sufficient for use as a quality monitoring tool. Repeatability of the results was much narrower, at less than 0.5% fat content, suggesting that the instrument holds the potential for a higher level of accuracy given further model development. The high performance of such models permitted to obtain reliable prediction maps which could be easily usable by the industry to sort fish according with the fat content required by the clients.
SPECTRAFISH offers multiple advantages:
• No sample preparation required.
• Fully automated.
• Non-invasive technology.
• Rapid data acquisition and analysis that will be capable of keeping pace with typical processing speeds.
• Quantification of nutrition and quality parameters, and visualization of their special distribution.

Potential Impact:
The SPECTRAFISH project has developed a novel hyperspectral imaging (HSI) system, for the rapid, non-invasive and simultaneous measurement of several safety and quality attributes of finfish.
The end-users of the technology will be able to differentiate their products and gain competitive advantage through the SPECTRAFISH benefits:
- Automated inspection of fish fillets
- Capable of online operation
- Rapid and non-invasive technology
- Simultaneous measurement of several parameters
- Ability to visualise the spatial distribution of parameters

By having access to a technology such as SPECTRAFISH, once it is ready for sell, manufacturers will be equipped to provide fish products that deliver on safety, quality and nutrition. The development of for rapid, objective, quantitative and non-invasive inspection system capable of visualising attribute distribution represents a very significant scientific advance for the finfish industry SPECTRAFISH will benefit both the European finfish industry and European consumers, through increased confidence in this product, together with a positive impact on the competitiveness of European companies along its supply chain.

In terms of Dissemination, a dissemination strategy has been in place to ensure that non-confidential SPECTRAFISH information is disseminated as widely as possible among industria stakeholders, academic community and general public. A project website ( was created and used to inform the public and interested parties on the basics of the technology, latest news and the progress of the project insofar as it can be disseminated without threatening the proper protection of the developed IP. Project poster and leaflet (annexed to the report) were prepared, printed and distributed in events, trade fairs and conferences where SMEs attended to disseminate the project to raise the awareness of SPECTRAFISH both in industry and in the public domain. In addition, the project has been disseminated among the scientific community via oral presentations and posters at conferences and through press releases targeted to general public and industry. Overall, 23 dissemination activities have been carried out throughout 26 months.

In addition, training activities were performed throughout project in order to ensure a profitable knowledge transfer from the RTDs to the technical and managerial staff of the industrial partners of the Consortium. Training activities consisted of a complementary approach based on written documentation, power point presentation, three on-site training sessions and a video clip for the Consortium members. Such activities were fundamental to facilitate acceptance and future uptake and exploitation of the results of the SPECTRAFISH project by the participant SMEs.

In terms of exploitation, the aspects of the foreground and their novelty have been presented and discussed during the project meetings based on the results of the RTD work. With increased expectations for food products with high quality and safety, the need for accurate, fast, and objective quality determination of these characteristics continues to grow, and as consequence the target market of SPECTRAFISH technology. The results obtained in the project are promising but at the final stage of the project, the SMEs´ understanding is that further development in terms of model development, is required to bring the SPECTRAFISH to a salable state. Nevertheless, results so far obtained indicate that the prototype could be suitable as a tool for measuring the composition of products of wide range of products.

List of Websites:

Dr Edurne Gaston Estanga