Christian Riess

Friedrich-Alexander-University of Erlangen-Nürnberg | FAU · Department of Computer Science

Learning-based algorithms for automated license plate recognition implicitly assume that the training and test data are well aligned. However, this may not be the case under extreme environmental conditions, or in forensic applications where the system cannot be trained for a specific acquisition device. Predictions on such out-of-distribution imag...

Handling entirely unknown data is a challenge for any deployed classifier. Classification models are typically trained on a static pre-defined dataset and are kept in the dark for the open unassigned feature space. As a result, they struggle to deal with out-of-distribution data during inference. Addressing this task on the class-level is termed op...

Freely available and easy-to-use audio editing tools make it straightforward to perform audio splicing. Convincing forgeries can be created by combining various speech samples from the same person. Detection of such splices is important both in the public sector when considering misinformation, and in a legal context to verify the integrity of evid...

Verifying the integrity of voice recording evidence for criminal investigations is an integral part of an audio forensic analyst's work. Here, one focus is on detecting deletion or insertion operations, so called audio splicing. While this is a rather easy approach to alter spoken statements, careful editing can yield quite convincing results. For...

Zusammenfassung Die Multimedia-Forensik ermöglicht Untersuchung von Spuren in digitalen Medieninhalten, um Authentizität und Ursprung der Inhalte zu ermitteln. Solche Verfahren können beispielsweise Journalisten bei der Faktenprüfung unterstützen. Im Zuge der ,,KI- Revolution‘‘ der letzten Jahre werden auch in der Multimedia-Forensik zunehmend Verf...

Combining multiple classifiers is a promising approach to hardening forensic detectors against adversarial evasion attacks. The key idea is that an attacker must fool all individual classifiers to evade detection. The 1.5C classifier is one of these multiple-classifier detectors that is attack-agnostic, and thus even increases the difficulty for an...

Hyperspectral remote sensing (HSRS) images have high dimensionality, and labeling HSRS data is expensive and therefore limited to small amounts of pixels. This makes it challenging to use deep neural networks for HSRS image classification. In extreme cases, deep neural networks are even outperformed by traditional models. In this work, we propose t...

An insufficient number of training samples is a common problem in neural network applications. While data augmentation methods require at least a minimum number of samples, we propose a novel, rendering-based pipeline for synthesizing annotated data sets. Our method does not modify existing samples but synthesizes entirely new samples. The proposed...

In this paper, we propose a data augmentation framework for Optical Character Recognition (OCR). The proposed framework is able to synthesize new viewing angles and illumination scenarios, effectively enriching any available OCR dataset. Its modular structure allows to be modified to match individual user requirements. The framework enables to comf...

Forensic license plate recognition (FLPR) remains an open challenge in legal contexts such as criminal investigations, where unreadable license plates (LPs) need to be deciphered from highly compressed and/or low resolution footage, e.g., from surveillance cameras. In this work, we propose a side-informed Transformer architecture that embeds knowle...

Freely available and easy-to-use audio editing tools make it straightforward to perform audio splicing. Convincing forgeries can be created by combining various speech samples from the same person. Detection of such splices is important both in the public sector when considering misinformation, and in a legal context to verify the integrity of evid...

One common task in image forensics is to detect spliced images, where multiple source images are composed to one output image. Most of the currently best performing splicing detectors leverage high-frequency artifacts. However, after an image underwent strong compression, most of the high frequency artifacts are not available anymore. In this work,...

Dynamic environments require adaptive applications. One particular machine learning problem in dynamic environments is open world recognition. It characterizes a continuously changing domain where only some classes are seen in one batch of the training data and such batches can only be learned incrementally. Open world recognition is a demanding ta...

Employing deep neural networks for Hyper-spectral remote sensing (HSRS) image classification is a challenging task. HSRS images have high dimensionality and a large number of channels with substantial redundancy between channels. In addition, the training data for classifying HSRS images is limited and the amount of available training data is much...

Physics-based methods anchor the forensic analysis in physical laws of image and video formation. The analysis is typically based on simplifying assumptions to make the forensic analysis tractable. In scenes that satisfy such assumptions, different types of forensic analysis can be performed. The two most widely used applications are the detection...

In many applications of forensic image analysis, state-of-the-art results are nowadays achieved with machine learning methods. However, concerns about their reliability and opaqueness raise the question whether such methods can be used in criminal investigations. So far, this question of legal compliance has hardly been discussed, also because lega...

One common task in image forensics is to detect spliced images, where multiple source images are composed to one output image. Most of the currently best performing splicing detectors leverage high-frequency artifacts. However, after an image underwent strong compression, most of the high frequency artifacts are not available anymore. In this work,...

Criminal investigations regularly involve the deciphering of license plates (LPs) of vehicles. Unfortunately, the image or video source material typically stems from uncontrolled sources, and may be subject to severe degradations such as extremely low resolution, strong compression, low contrast or over- resp. underexposure. While LP recognition ha...

Purpose In Talbot–Lau X-ray phase contrast imaging, the measured phase value depends on the position of the object in the measurement setup. When imaging large objects, this may lead to inhomogeneous phase contributions within the object. These inhomogeneities introduce artifacts in tomographic reconstructions of the object. Methods In this work,...

High resolution electroluminescence (EL) images captured in the infrared spectrum allow to visually and non-destructively inspect the quality of photovoltaic (PV) modules. Currently, however, such a visual inspection requires trained experts to discern different kinds of defects, which is time-consuming and expensive. Automated segmentation of cell...

Identifying the model of a camera that has captured an image can be an important task in criminal investigations. Many methods assume that the image under analysis originates from a given set of known camera models. In practice, however, a photo can come from an unknown camera model, or its appearance could have been altered by unknown post-process...

License plate recognition is instrumental in many forensic investigations involving organized crime and gang crime, burglaries and trafficking of illicit goods or persons. After an incident, recordings are collected by police officers from cameras in-the-wild at gas stations or public facilities. In such an uncontrolled environment, a generally low...

Image provenance can represent crucial knowledge in criminal investigation and journalistic fact checking. In the last two decades, numerous algorithms have been proposed for obtaining information on the source camera and distribution history of an image. For a fair ranking of these techniques, it is important to rigorously assess their performance...

Recent advances in deep learning have led to significant improvements in single image super-resolution (SR) research. However, due to the amplification of noise during the upsampling steps, state-of-the-art methods often fail at reconstructing high-resolution images from noisy versions of their low-resolution counterparts. However, this is especial...

X-ray phase-contrast imaging enhances soft-tissue contrast. The measured differential phase signal strength in a Talbot-Lau interferometer is dependent on the object's position within the setup. For large objects, this affects the tomographic reconstruction and leads to artifacts and perturbed phase values. In this paper, we propose a pipeline to l...

Supervised machine learning requires a large amount of labeled data to achieve proper test results. However, generating accurately labeled segmentation maps on remote sensing imagery, including images from synthetic aperture radar (SAR), is tedious and highly subjective. In this work, we propose to alleviate the issue of limited training data by ge...

This volume discusses media ethics perspectives on truth in the context of digitalisation, while also addressing loss of trust and interpretations of the truth in public communication. It develops theoretical classifications of ‘fake news’ and disinformation from both a sociological and a media philosophy perspective. Empirical investigations and c...

High dynamic range (HDR) imaging combines multiple images with different exposure times into a single high-quality image. The image signal processing pipeline (ISP) is a core component in digital cameras to perform these operations. It includes demosaicing of raw color filter array (CFA) data at different exposure times, alignment of the exposures,...

Image provenance can represent crucial knowledge in criminal investigation and journalistic fact checking. In the last two decades, numerous algorithms have been proposed for obtaining information on the source camera and distribution history of an image. For a fair ranking of these techniques, it is important to rigorously assess their performance...

The reconstruction problem of voxels with individual weightings can be modeled a position- and angle- dependent function in the forward-projection. This changes the system matrix and prohibits to use standard filtered backprojection. In this work we first formulate this reconstruction problem in terms of a system matrix and weighting part. We compu...

In multimedia forensics, learning-based methods provide state-of-the-art performance in determining origin and authenticity of images and videos. However, most existing methods are challenged by out-of-distribution data, i.e., with characteristics that are not covered in the training set. This makes it difficult to know when to trust a model, parti...

Browser fingerprinting as a tracking technique to recognize users based on their browsers' unique features or behavior has been known for more than a decade. We present the results of a 3-year online study on browser fingerprinting with more than 1,300 users. This is the first study with ground truth on user level, which allows the assessment of tr...

Image provenance, i.e., information on the model and make of the device that was used to produce an image, is a helpful cue in many digital investigations. Such information can, for example, help refute the hypothesis that an illegal photograph found on the Internet was produced using the personal device of a suspect. Grouping images by provenance...

An X-ray Talbot-Lau grating interferometer enables imaging of X-ray phase contrast. It can measure material transitions that are difficult to observe with traditional attenuation X-ray, for example soft-tissue variations in medical diagnosis or weakly absorbing materials in optical inspection. Unfortunately, the field of view of a Talbot-Lau interf...

The image signal processing pipeline (ISP) is a core element of digital cameras to capture high-quality displayable images from raw data. In high dynamic range (HDR) imaging, ISPs include steps like demosaicing of raw color filter array (CFA) data at different exposure times, alignment of the exposures, conversion to HDR domain, and exposure mergin...

The goal of blind image forensics is to determine authenticity and origin of an image without using an explicitly embedded security scheme. Most existing forensic methods can roughly be grouped into statistical and physics-based approaches. Statistical methods can oftentimes be fully automated, and achieve impressive results on current state-of-the...

The JPEG compression format provides a rich source of forensic traces that include quantization artifacts, fingerprints of the container format, and numerical particularities of JPEG compressors. Such a diverse set of cues serves as the basis for a forensic examiner to determine origin and authenticity of an image. In this work, we present a novel...

The X-ray dark-field signal can be measured with a grating-based Talbot-Lau interferometer. It measures small angle scattering of micrometer-sized oriented structures. Interestingly, the signal is a function not only of the material, but also of the relative orientation of the sample, the X-ray beam direction, and the direction of the interferomete...

In hyperspectral remote sensing (HSRS), feature data can potentially become very high dimensional. At the same time, manual labeling of that data is an expensive task. As a consequence of these two factors, one of the core challenges is to perform multi-class classification using only relatively few training data points. In this work, we investigat...

Electroluminescence (EL) imaging is a useful modality for the inspection of photovoltaic (PV) modules. EL images provide high spatial resolution, which makes it possible to detect even finest defects on the surface of PV modules. However, the analysis of EL images is typically a manual process that is expensive, time-consuming, and requires expert...

Capturing ground truth data to benchmark super-resolution (SR) is challenging. Therefore, current quantitative studies are mainly evaluated on simulated data artificially sampled from ground truth images. We argue that such evaluations overestimate the actual performance of SR methods compared to their behavior on real images. Toward bridging this...

This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and n...

A common problem in forensic investigations is the identification of the source of multimedia data, i.e., determining the model, make or individual device that recorded media content. In contrast to methods based on sensor noise, source linkage based on header information of media items allows for easy automation. Such header information involves m...

Compared to conventional attenuation x-ray radiographic imaging, the x-ray Talbot-Lau technique provides further information about the scattering and the refractive properties of the object in the beam path. Hence, this additional information should improve the diagnostic process concerning medical applications and non-destructive testing. Neverthe...

X-ray Phase-Contrast Imaging (PCI) is a novel imaging technique that can be implemented with an grating interferometer. PCI is compatible with clinical X-ray equipment, and yields in addition to an absorption image also a differential phase image and a dark- field image. Computed Tomography (CT) of the differential phase can in principle provide hi...

The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns on the implication on the society. At best, this leads to a loss of trust in digital content, but it might even cause further harm by spreading false information and the creation of fake news. In this paper, we examine the...

Distinguishing fakes from real images is becoming increasingly difficult as new sophisticated image manipulation approaches come out by the day. Convolutional neural networks (CNN) show excellent performance in detecting image manipulations when they are trained on a specific forgery method. However, on examples from unseen manipulation approaches,...

Talbot-Lau X-ray phase-contrast imaging is a novel imaging modality, which provides not only an X-ray absorption image, but also additionally a differential phase image and a dark-field image. The dark-field image is related to small angle scattering and has an interesting property when canning oriented structures: the recorded signal depends on th...

Purpose Two phase gratings in an X-ray grating interferometers can solve several technical challenges for clinical use of X-ray phase contrast. In this work, we adapt and evaluate this setup design to clinical X-ray sources and detectors in a simulation study. Methods For a given set of gratings, we optimize the remaining parameter space of a dual...

This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and n...

Smart heating applications promise to increase energy efficiency and comfort by collecting and processing room climate data. While it has been suspected that the sensed data may leak crucial personal information about the occupants, this belief has up until now not been supported by evidence. In this work, we investigate privacy risks arising from...

Capturing ground truth data to benchmark super-resolution (SR) is challenging. Therefore, current quantitative studies are mainly evaluated on simulated data artificially sampled from ground truth images. We argue that such evaluations overestimate the actual performance of SR methods compared to their behavior on real images. To bridge this simula...

X-Ray Phase-Contrast Imaging (PCI) yields absorption, differential phase, and dark-field images. Computed Tomography (CT) of grating-based PCI can in principle provide high-resolution soft-tissue contrast. Recently, grating-based PCI took several hurdles towards clinical implementation by addressing, for example, acquisition speed, high X-ray energ...

Modern medical imaging is achieved with state-of-the-art devices, that use cutting-edge technology from different fields of engineering. Oftentimes, progress is driven by discoveries in a at first sight seemingly unrelated field of research.

Electroluminescence (EL) imaging is a useful modality for the inspection of photovoltaic (PV) modules. EL images provide high spatial resolution, which makes it possible to detect even finest defects on the surface of PV modules. However, the analysis of EL images is typically a manual process that is expensive, time-consuming, and requires expert...

High resolution electroluminescence (EL) images captured in the infrared spectrum allow to visually and non-destructively inspect the quality of photovoltaic (PV) modules. Currently, however, such a visual inspection requires trained experts to discern different kind of defects, which is time-consuming and expensive. In this work, we make an import...

X-Ray Phase-Contrast Imaging (PCI) yields absorption, differential phase, and dark-field images. Computed Tomography (CT) of grating-based PCI can in principle provide high-resolution soft-tissue contrast. Recently, grating-based PCI took several hurdles towards clinical implementation by addressing, for example, acquisition speed, high X-ray energ...

Inverse problems play a central role for many classical computer vision and image processing tasks. A key challenge in solving an inverse problem is to find an appropriate prior to convert an ill-posed problem into a well-posed task. Many of the existing priors, like total variation, are based on ad-hoc assumptions that have difficulties to represe...

With recent advances in computer vision and graphics, it is now possible to generate videos with extremely realistic synthetic faces, even in real time. Countless applications are possible, some of which raise a legitimate alarm, calling for reliable detectors of fake videos. In fact, distinguishing between original and manipulated video can be a c...

X-ray grating-based phase-contrast imaging has raised interest regarding a variety of potential clinical applications, whereas the method is feasible using a medical x-ray tube. Yet, the transition towards a clinical setup remains challenging due to the requirement of mechanical robustness of the interferometer and high demands applying to medical...

Three-dimensional camera systems are useful sensors for several higher level vision tasks like navigation, environment mapping or dimensioning. However, the raw 3-D data is for many algorithms not the best representation. Instead, many methods rely on a more abstract scene description, where the scene is represented as a collection of geometric pri...

Old master drawings were mostly created step by step in several layers using different materials. To art historians and restorers, examination of these layers brings various insights into the artistic work process and helps to answer questions about the object, its attribution and its authenticity. However, these layers typically overlap and are of...

The amount of training data that is required to train a classifier scales with the dimensionality of the feature data. In hyperspectral remote sensing, feature data can potentially become very high dimensional. However, the amount of training data is oftentimes limited. Thus, one of the core challenges in hyperspectral remote sensing is how to perf...

High-resolution imaging has delivered new prospects for detecting the material composition and structure of cultural treasures. Despite the various techniques for analysis, a significant diagnostic gap remained in the range of available research capabilities for works on paper. Old master drawings were mostly composed in a multi-step manner with va...

In this work, we conducted a survey on different registration algorithms and investigated their suitability for hyperspectral historical image registration applications. After the evaluation of different algorithms, we choose an intensity based registration algorithm with a curved transformation model. For the transformation model, we select cubic...

Multi-modal sensory data plays an important role in many computer vision and robotics tasks. One popular multi-modal pair is cameras and laser scanners. To overlay and jointly use the data from both modalities, it is necessary to calibrate the sensors, i.e., to obtain the spatial relation between the sensors. Computing such a calibration is challen...

Smart heating applications promise to increase energy efficiency and comfort by collecting and processing room climate data. While it has been suspected that the sensed data may leak crucial personal information about the occupants , this belief has up until now not been supported by evidence. In this work, we investigate privacy risks arising from...

Over the past decades, various super-resolution (SR) techniques have been developed to enhance the spatial resolution of digital images. Despite the great number of methodical contributions, there is still a lack of comparative validations of SR under practical conditions, as capturing real ground truth data is a challenging task. Therefore, curren...

Illumination direction and color are two physics-based forensic cues that are based on the same underlying model. In this work, we discuss these methods in the light of their joint physical model, with a particular focus on the limitations and a qualitative study of failure cases of these methods. Our goal is to provide directions for future resear...

Grating-based Talbot-Lau x-ray interferometry is a popular method for measuring absorption, phase shift, and small-angle scattering. The standard acquisition method for this modality is phase stepping, where the Talbot pattern is reconstructed from multiple images acquired at different grating positions. We review the implicit assumptions in phase-...

In cone-beam CT, involuntary patient motion and inaccurate or irreproducible scanner motion substantially degrades image quality. To avoid artifacts this motion needs to be estimated and compensated during image reconstruction. In previous work we showed that Fourier Consistency Conditions (FCC) can be used in fan-beam CT to estimate motion in the...

We report on a radiographic measurement of an ex vivo human knee using a grating-based phase-contrast imaging setup and a medical x-ray tube at a tube voltage of 70 kV. The measurement has been carried out using a Talbot–Lau setup that is suitable to achieve a high visibility in the energy regime of medical imaging. In a medical reading by an exper...