Next generation HSI

Hyperspectral imaging can be used for more than remote sensing according to Dr John Ferguson, we find out more

ANYONE familiar with hyperspectral imaging (HSI) will probably think of military reconnaissance and surveillance and environmental remote sensing but whilst these were amongst the first uses and are still critically important, the technology is now finding a host of other, more “down to earth” applications.

A typical hyperspectral imager consists of a camera lens, mounted in front of the entrance slit of a diffraction-based spectrometer, together with a high sensitivity camera in the form of a 2-D detector array such as a CCD. This is illustrated schematically in Figure 1. The light reflected from an object across a broad region of the electromagnetic spectrum is captured and split into a large number of spectral bands, typically hundreds, and this combination of imaging and spectroscopy allows both the spatial and spectral information of an object or scene to be acquired. Only a small “slice” of the scene is recorded at a time so it is necessary to scan the imager across the scene, or in some cases, move the object across the imager’s field of view. On completion of the scan, each of the detector’s pixels has acquired light intensity and spectral data which is usually illustrated as a 3-D cube or “hypercube”, with the X and Y coordinates representing the spatial information and Z, the wavelength. This data can be analysed and interpreted in several different ways which will largely be governed by the application. Many commercial software packages are available and a hyperspectral data manipulating programme is freely available from the US Army Geospatial Centre (see: www.agc.army.mil/hypercube/).

Hyperspectral techniques clearly yield far more detailed spectral information about a scene or object than a normal colour camera – which generally only acquires three channels corresponding to the visual primary colours – and although multispectral imaging offers more channels at selected wavebands and covers some additional (non-visible) wavelengths, this technique also offers far lower spectral resolution than HSI. Hence, the use of HSI leads to a vastly improved ability to characterise objects based on their spectral properties. This has led to the technique finding an ever-growing number of applications which has been made possible by critical technological developments such as aberration-corrected optics, application-specific designs and improved software. These applications, which include airborne, field, in-line, on-process and laboratory uses, range from the traditional – such as military surveillance, mineral exploration and environmental monitoring – to industrial quality assurance and inspection, food safety, forensics, healthcare and medical research. A selection of these newer applications is listed in Table 1.

A prime example of where HSI can play an unexpected role is in the poultry industry, where the detection of faecal contamination is vital to protect public health. The American Agricultural Research Service (ARS) has recently developed a system which aims to automate what has traditionally been a time-consuming process that is prone to human error: to detect small masses (i.e., 2-10 mg) of faecal contaminants on broiler carcasses. Trials were conducted with both multispectral and hyperspectral imaging techniques. In the latter, a Hyperspec imager produced by Headwall Photonics was used which operated in the 400-1000 nm region.

On each of three sampling days, 24 eviscerated, pre-chilled broiler carcasses were collected from a commercial processing plant. Caecal (i.e. intestinal) contents from the same flock were also collected and used to contaminate the carcasses. Carcass halves were first imaged uncontaminated and again after the caecal contents (2, 5 or 10 mg) had been applied. Contaminants were predicted by decision tree (DT) and mixture tuned matched filter (MTMF) classifiers. The DT classifier, applied to the multispectral imagery, detected 63%, 80% and 100% of the caecal mass applied at about 2, 5 and 10 mg, respectively. However, the MTMF classifier, applied to the hyperspectral data, detected 88% of 2 mg caeca and 100% of the 5 mg and 10 mg contaminants. Trials have demonstrated that the technique is able to detect faecal contaminants when operated at a commercial processing speed of 140 birds per minute.

Related work by the ARS has shown that HSI can be used in the laboratory to identify Campylobacter – a food-borne pathogen causing diarrhoea – on agar plate cultures. A system operating in the visible and near-infrared (NIR) spectral region (400-900 nm), used with a classification algorithm, was able to locate and identify Campylobacter, non-Campylobacter contaminants and background agars with an accuracy of 99.29%. The fully developed imaging system is expected to locate automatically and identify micro-organisms grown on Petri dishes and has a potential to be expanded to detect other pathogens such as E. coli and Salmonella grown on agar media. The ARS plans to investigate further uses of HSI including the detection of embedded bones in chicken breast fillets, the identification of defects and contaminants in eggs and the detection and identification of bacterial and chemical contaminants in meat products.

HSI techniques are starting to play a growing role in the life sciences; a field which

is likely to represent the next major wave of applications. For example, when incorporated into microscopes, HSI can be used to examine conventionally stained slides viewed in transmission mode and have shown promise in detecting staining alterations (metachromasia) resulting from cancer-related changes in a cell’s macromolecular components. Systems can also be used to quantify light-absorbing chromophores such as haemoglobin and bilirubin. Clinically, such approaches can be used to confirm and quantify the presence of abnormal tissue on a conventional pathology slide or to clarify tumour margins in surgical pathology.

Of specific interest for drug development are in vivo techniques, which permit real-time visualisation of the take-up and effectiveness of therapeutic agents, particularly when they are bound to nanoparticles or quantum dots. Confocal fluorescence microscopy is an excellent method for localising pigments in cells, as long as there is little spectral overlap between different fluorescing pigments. The application of HSI to fluorescence imaging is extending the capabilities of this technique by identifying pigments with similar fluorescent spectra. In the clinical context, automated imaging platforms incorporating conventional optical microscopes with HSI systems and intelligent software have the potential to transform diagnostic medicine, as more and more diagnostically and therapeutically relevant targets are identified.

Indeed, HSI technology has great potential in the pharmaceuticals industry. In a post-discovery production environment, capturing precise spectral information from manufacturing control points has traditionally involved either simple machine vision or single-point NIR spectral instruments deployed off-line. The limitation is that these systems are only capable of sampling a very small area of the overall product flow and do not lend themselves to use in high speed production processes. When combined with spectral libraries established during the drug discovery phase and multivariate analytical models, hyperspectral sensors such as Headwall Photonics’ Hyperspec NIR (900-1700 nm) or the Hyperspec SWIR (1000-2500 nm) can make “accept or reject” decisions when deployed on- or at-line. As a result, HSI allows pharmaceutical manufacturers to establish critical control points from the post-discovery phase through pre-production to high volume manufacture.

Further, as hyperspectral imagers create a wavelength intensity map with high spatial resolution, users can, for example, perform high-speed analysis of tablets’ chemical content; monitor content uniformity; and control the quality of spray dry dispersion. Most importantly, at a time when an ever-growing number of counterfeit drugs are entering the healthcare system, HSI can be used to verify and authenticate pharmaceutical products. This is critical, as the World Health Organisation estimates that around 10% of pharmaceuticals on the global market are counterfeit and in parts of the developing world, this figure is believed to be considerably higher.

Forensic laboratories are making growing use of HSI systems in applications such as the examination of legal documents and currency, where verification and authentication is clearly crucial. The analysis of inks and pigments, in particular, can reveal document alterations and forgeries. Since the spectral reflectance curve of every pixel on the entire measured area is recorded, it is not necessary to make a pre-selection of the ink or pigment being investigated. Studies have shown that HSI at visible/NIR wavelengths can discriminate between black ballpoint pair samples which were indistinguishable when examined by conventional video spectral analysis. The FBI has recently used a commercial HSI system to identify counterfeit currency and fraudulent passports as well as to examine questioned documents. Headwall’s Hyperspec Starter Kit is well suited to this type of laboratory application, as it may be equipped with a variable speed linear motion stage and offers variable illumination intensity and angle, allowing the rapid and detailed scanning of documents, banknotes and other samples.

Clearly, HSI systems are highly application-specific but major manufacturers can provide prospective users with advice on how to select the product best suited to their needs.

A critical issue is whether the system uses transmissive or reflective optics. Although the former is widely used, the latter approach reduces chromatic aberration and stray light, and thus system noise, leading to improved signal-to-noise ratios and a better dynamic range. A key factor to consider is the spectral resolution, i.e., the separation between adjacent spectral lines, which determines the spectral detail and the amount of data captured. In some systems it is possible to change the width of the spectrometer’s entrance slit which alters the spectral resolution. Resolution is also governed by the linear dispersion of the spectrometer and the pixel size on the detector array. In addition to resolution, slit width also influences scanning speed and the area resolved during a scan. Interchangeable slits therefore considerably enhance a product’s versatility. Two terms likely to be encountered in manufacturers’ literature are “smile” and “keystone”. These effects represent the slight curvatures produced by the optical elements in two-dimensional spectral image detectors. In a perfect detector, the spectral and spatial information would be orthogonally projected onto the detector matrix. Smile refers to the curvature of the spectral dispersion data; keystone refers to the curvature of the spatial information. These should both be minimal. The operating wavelength will be governed by the application and manufacturers offer instruments with ranges that extend from the UV/visible – from ~330 nm and above – through the NIR into the mid-IR at ~2500 nm. An example of a typical product range is: 330-830 nm, 400-1000 nm, 600-1600 nm, 900-1700 nm and 1000-2500 nm.

Recent years have seen HSI evolve from being principally an airborne technique to a versatile tool with wide-ranging applications in laboratories and production environments. Today’s systems are smaller, more rugged and easier to use than their predecessors and also offer improved performance. However, the technology is far from static and manufacturers are constantly seeking improvements. In the future we can expect to see size reductions, improved data analysis techniques and operating wavelengths extending out to the far IR.

Table 1: Some applications of hyperspectral imaging