## In Review

 T. W. Rogers, N. Jaccard, E. J. Morton and L. D. Griffin A Deep Learning Framework for the Automated Inspection of Complex Dual-Energy X-ray Cargo Imagery Submitted to SPIE Anomaly Detection and Imaging with X-Rays (ADIX) T. W. Rogers, N. Jaccard, E. J. Morton and L. D. Griffin Automated Detection of Loads in Cargo Containers Submitted to IEEE Transactions on Cybernetics

## Publications

#### Nb: Items with a symbol can be expanded for extra details.

N. Jaccard, T. W. Rogers, E. J. Morton and L. D. Griffin

Detection of concealed cars in complex cargo X-ray imagery using deep learning

Journal of X-ray Science and Technology (accepted) [arXiv]

N. Jaccard, T. W. Rogers, E. J. Morton and L. D. Griffin

Automated detection of smuggled high-risk security threats using Deep Learning

International Conference on Imaging for Crime Detection and Prevention [arXiv]

T. W. Rogers, N. Jaccard, E. J. Morton and L. D. Griffin

Automated X-ray Image Analysis for Cargo Security: Critical Review and Future Promise

Journal of X-ray Science and Technology (2016).

## Automated X-ray Image Analysis for Cargo Security: Critical Review and Future Promise

Abstract: We review the relatively immature field of automated image analysis for X-ray cargo imagery. There is increasing demand for automated analysis methods that can assist in the inspection and selection of containers, due to the ever-growing volumes of traded cargo and the increasing concerns that customs- and security-related threats are being smuggled across borders by organised crime and terrorist networks. We split the field into the classical pipeline of image preprocessing and image understanding. Preprocessing includes: image manipulation; quality improvement; Threat Image Projection (TIP); and material discrimination and segmentation. Image understanding includes: Automated Threat Detection (ATD); and Automated Contents Verification (ACV). We identify several gaps in the literature that need to be addressed and propose ideas for future research. Where the current literature is sparse we borrow from the single-view, multi-view, and CT X-ray baggage domains, which have some characteristics in common with X-ray cargo.

T. W. Rogers, J. Ollier, E. J. Morton and L. D. Griffin

Measuring and correcting wobble in large-scale transmission radiography

Journal of X-ray Science and Technology (2016, accepted).

## Measuring and correcting wobble in large-scale transmission radiography

BACKGROUND: Large-scale transmission radiography scanners are used to image vehicles and cargo containers. Acquired images are inspected for threats by a human operator or a computer algorithm. To make accurate detections, it is important that image values are precise. However, due to the scale (~5m tall) of such systems, they can be mechanically unstable, causing the imaging array to wobble during a scan. This leads to an effective loss of precision in the captured image. OBJECTIVE: We consider the measurement of wobble and amelioration of the consequent loss of image precision. METHODS: Following our previous work, we use Beam Position Detectors (BPDs) to measure the cross-sectional profile of the X-ray beam, allowing for estimation, and thus correction, of wobble. We propose: (i) a model of image formation with a wobbling detector array; (ii) a method of wobble correction derived from this model; (iii) methods for calibrating sensor sensitivities and relative offsets; (iv) a Random Regression Forest based method for instantaneous estimation of detector wobble; and (v) using these estimates to apply corrections to captured images of difficult scenes. RESULTS: We show that these methods are able to correct for 87% of image error due wobble, and when applied to difficult images, a significant visible improvement in the intensity-windowed image quality is observed. CONCLUSIONS: The method improves the precision of wobble affected images, which should help improve detection of threats and the identification of different materials in the image.

T. W. Rogers, N. Jaccard, E. D. Protonotarios, J. Ollier, E. J. Morton and L. D. Griffin

Threat Image Projection (TIP) into X-ray images of cargo containers for training humans and machines

50th IEEE International Carnahan Conference on Security Technology (2016, accepted).

## Threat Image Projection (TIP) into X-ray images of cargo containers for training humans and machines

Abstract: We propose a framework for Threat Image Projection (TIP) in cargo transmission X-ray imagery. The method exploits the approximately multiplicative nature of X-ray imagery to extract a library of threat items. These items can then be projected into real cargo. We show using experimental data that there is no significant qualitative or quantitative difference between real threat images and TIP images. We also describe methods for adding realistic variation to TIP images in order to robustify Machine Learning (ML) based algorithms trained on TIP. These variations are derived from cargo X-ray image formation, and include: (i) translations; (ii) magnification; (iii) rotations; (iv) noise; (v) illumination; (vi) volume and density; and (vii) obscuration. These methods are particularly relevant for representation learning, since it allows the system to learn features that are invariant to these variations. The framework also allows efficient addition of new or emerging threats to a detection system, which is important if time is critical.

We have applied the framework to training ML-based cargo algorithms for (i) detection of loads (empty verification), (ii) detection of concealed cars (ii) detection of Small Metallic Threats (SMTs). TIP also enables algorithm testing under controlled conditions, allowing one to gain a deeper understanding of performance. Whilst we have focused on robustifying ML-based threat detectors, our TIP method can also be used to train and robustify human threat detectors as is done in cabin baggage screening.

N. Jaccard, T. W. Rogers, E. J. Morton, and Lewis D. Griffin

Tackling the X-ray cargo inspection challenge using machine learning

Proc. SPIE 9847, Anomaly Detection and Imaging with X-Rays (ADIX), 98470N (2016).

## Tackling the X-ray cargo inspection challenge using machine learning

Abstract: The current infrastructure for non-intrusive inspection of cargo containers cannot accommodate exploding commerce volumes and increasingly stringent regulations. There is a pressing need to develop methods to automate parts of the inspection work-flow, enabling expert operators to focus on a manageable number of high-risk images. To tackle this challenge, we developed a modular framework for automated X-ray cargo image inspection. Employing state-of-the-art machine learning approaches, including deep learning, we demonstrate high performance for empty container verification and specific threat detection. This work constitutes a significant step towards the partial automation of X-ray cargo image inspection.

N. Jaccard, T. W. Rogers, E. J. Morton, and Lewis D. Griffin

Using deep learning on X-ray images to detect threats

Defence and Security Doctoral Symposium, Cranfield University (2015).

## Using deep learning on X-ray images to detect threats

Shortened Abstract: World trade volumes are exploding, with cargo containers totaling an estimated 500 million TEU (Twenty-foot equivalent units) shipped globally in 2012. At the same time, security requirements and transport regulations are increasingly stringent, putting significant pressure on the infrastructure at transport hubs and borders. In order to meet ambitious aims set by authorities, such as the inspection of every US-bound container, there is a pressing need to devise procedures to cope with high trade volumes while minimising the impact on the stream-of-commerce (SoC) ... We have developed a deep learning framework for the classification of X-ray cargo images according to their content. This framework is based on convolutional neural networks (CNNs), a class of artificial neural networks that currently is the state-of-the-art in many areas of machine-learning and –vision ... CNNs typically require very large training datasets, the acquisition of which is costly and impractical for cargo images ... In this contribution, we present an overview of our deep learning framework and present preliminary results, including a comparison to a more conventional approach we previously proposed for object detection in X-ray cargo container images. While our focus has been on cargo containers, we expect the framework to generalise to X-ray images of other vehicles and luggage. As such, this research is expected to contribute to the development of specialised software packages to assist operators through partial automation of the inspection work-flow.

T. W. Rogers, N. Jaccard, E. J. Morton, and Lewis D. Griffin

Detection of cargo container loads from X-ray images

IET Intelligent Signal Processing (2015).

## Detection of cargo container loads from X-ray images

Abstract: Over 100 million cargo containers that are declared empty on their manifests are transported globally each year. Human operators can confirm if each is truly empty by physical inspection or by examination of an X-ray image. However, the huge number transported means that confirmation is far from complete. Thus, empty containers offer an opportunity for criminals to smuggle contraband. We report an algorithm for automatically detecting loads in cargo containers from transmission X-ray images. Detection without generation of excessive false positives is complicated by the fabric of the container, container variation, damage, and detritus. The algorithm detects 99.3% of loads in stream-of-commerce date while raising false alarms on 0.7% of actually empty containers. On challenging data, created by image synthesis, we are able to achieve 90% detection of loads with the same size and attenuation as a 1.5 kg cube of cocaine or 1 L of water, while triggering fewer than 1-in-605 or 1-in-197 false alarms respectively, on truly empty containers. The algorithm analyses each small window of the image separately, and detects loads within the window by random forest classification of texture features together with the window coordinates.

N. Jaccard, T. W. Rogers, E. J. Morton, and Lewis D. Griffin

Automated detection of cars in transmission X-ray images of freight containers

IEEE Advanced Video and Signal Based Surveillance (2014).

## Automated detection of cars in transmission X-ray images of freight containers

Abstract: We present a method for automated car detection in xraytransmission images of freight containers. A random forest classifier was used to classify image sub-windows as “car” and “non-car” based on image features such as intensity and log-intensity, as well as local structures and symmetries as encoded by Basic Image Features (BIFs) and oriented Basic Image Features (oBIFs). The proposed approach was validated using a dataset of stream of commerce X-ray images. A car detection rate of 100% was achieved while maintaining a false alarm rate of 1.23%. Further reduction in false alarm rate, potentially at the cost of detection rate, was possible by tweaking the classification confidence threshold. This work establishes a framework for the automated classification of X-ray transmission cargo images and their content, paving the way towards the development of tools to assist custom officers faced with an ever increasing number of images to inspect.

T. W. Rogers, J. Ollier, E. J. Morton, and Lewis D. Griffin

Reduction of wobble artefacts in images from mobile transmission X-ray vehicle scanners

IEEE Imaging Systems and Techniques (2014).

## Reduction of wobble artefacts in images from mobile transmission X-ray vehicle scanners

Abstract: Detector boom wobble in transmission X-ray vehicle scanners is an unpredictable and currently uncontrollable problem, which lowers the quality of captured X-ray images. We propose (i) a method for image correction which is able to correct for 70% of boom wobble error given estimates of boom wobble, and (ii) a method of wobble estimation, based on the fusion of instantaneous wobble estimates with previous estimates, which is robust against non-Gaussian X-ray beam cross-sections and approaches ground truth accuracy. The combination of the two approaches provides a method for the reduction of wobble artefacts in images. The two methods have good potential for application in analogous scenarios in medical imaging, radiation physics, laser science and biophysics.

N. S. Blunt, T. W. Rogers, J. S. Spencer, and W. M. C. Foulkes

Density-matrix quantum Monte Carlo method

Physical Review B 89, 245124 (2014).

## Density-matrix quantum Monte Carlo method

Abstract: We present a quantum Monte Carlo method capable of sampling the full density matrix of a many-particle system at finite temperature. This allows arbitrary reduced density matrix elements and expectation values of complicated nonlocal observables to be evaluated easily. The method resembles full configuration interaction quantum Monte Carlo but works in the space of many-particle operators instead of the space of many-particle wave functions. One simulation provides the density matrix at all temperatures simultaneously, from $T=\infty$ to $T=0$, allowing the temperature dependence of expectation values to be studied. The direct sampling of the density matrix also allows the calculation of some previously inaccessible entanglement measures. We explain the theory underlying the method, describe the algorithm, and introduce an importance-sampling procedure to improve the stochastic efficiency. To demonstrate the potential of our approach, the energy and staggered magnetization of the isotropic antiferromagnetic Heisenberg model on small lattices, the concurrence of one-dimensional spin rings, and the Renyi ${S}_{2}$ entanglement entropy of various sublattices of the $6×6$ Heisenberg model are calculated. The nature of the sign problem in the method is also investigated.