Automated Analysis Of Coastal Webcam Footage By Means Of Machine Learning

Abstract

This thesis aims to create a machine-learning approach for predicting wave heights and water levels from webcam images. For this purpose, different models and input data types were tested. Classification and regression tasks were performed with deep learning and machine learning models. The water level prediction was divided into four models depending on the sea state visible in the models. There are models for calm, smooth, slight, and rough seas and a model for wave height prediction. The best results were achieved using supervised machine learning with Gaussian process regression. For the 2015 dataset, the wave height prediction achieved an error of 11 cm while the water level prediction achieved a mean error of 26 cm. The errors for the individual water level prediction models were 0 cm for calm seas, 14 cm for smooth seas, 12 cm for slight seas, and 1 cm for moderate seas.

Full text

Studiengang
Marine Sensorik (M.Sc.)
MASTERARBEIT
Titel
Automated Analysis Of Coastal Webcam Footage By
Means Of Machine Learning
vorgelegt von
Julia Steiwer
Betreuender Gutachter
Prof. Dr. Oliver Zielinski
Zweiter Gutachter
Prof. Dr.-Ing. habil. Torsten Schlurmann
Oldenburg, 02.02.2023

Abstract
ENGLISH VERSION
This thesis aims to create a machine-learning approach for predicting wave heights and
water levels from webcam images. For this purpose, different models and input data types
were tested. Classification and regression tasks were performed with deep learning and
machine learning models. The water level prediction was divided into four models
depending on the sea state visible in the models. There are models for calm, smooth, slight,
and rough seas and a model for wave height prediction. The best results were achieved using
supervised machine learning with Gaussian process regression. For the 2015 dataset, the
wave height prediction achieved an error of 11 cm while the water level prediction achieved
a mean error of 26 cm. The errors for the individual water level prediction models were 0 cm
for calm seas, 14 cm for smooth seas, 12 cm for slight seas, and 1 cm for moderate seas.
DEUTSCHE VERSION
Ziel dieser Masterarbeit war es, einen Ansatz basierend auf maschinellem Lernen zu
entwickeln, der die Parameter Wellenhöhe und Wasserstand aus Webcam-Bildern
vorhersagen kann. Zu diesem Zweck wurden verschiedene Modelle und Datentypen getestet.
Klassifizierung und Regression wurden mit Modellen aus den Bereichen Tiefes und
Maschinelles Lernen durchgeführt. Die Vorhersage des Wasserstands wurde in vier Modelle
nach sichtbarem Wellengang auf den Bildern unterteilt. Es gibt Modelle für glatt-ruhige,
schwach, leicht und mäßig bewegte See, sowie ein Modell für die Bestimmung der
Wellenhöhe. Die besten Ergebnisse wurden mit überwachtem maschinellen Lernen und
Gaußprozess-Regression erreicht. Auf dem Datensatz von 2015 erreichte die Vorhersage der
Wellenhöhe einen Fehler von 11 cm, während der mittlere Fehler für den gesamten
Wasserstand bei 26 cm lag. Für die einzelnen Vorhersagemodelle des Wasserstands lagen
folgende Fehler vor: 0 cm für glatt-ruhige See, 14 cm für schwach bewegte See, 12 cm für
leicht bewegte See, und 1 cm für mäßig bewegte See.
1

Contents
Abstract 1
Contents 2
List of Figures 4
List of Tables 10
List of Abbreviations 11
1. Introduction 13
2. Literature Review 17
2.1 Study Site 17
2.2 Preliminary Work 23
2.3 Machine and Deep Learning 32
2.3.1 Background Information 32
2.3.2 Application in Coastal Engineering 35
3. Materials and Methods 42
3.1 Webcam Technology 42
3.2 Data Sources 46
3.2.1 Wave Data 46
3.2.2 Water Level 48
3.3 Processing and Formatting Data 50
3.3.1 Preprocessing the Image Data 50
3.3.2 Creating Regression Data 60
3.3.3 Preprocessing Image Data for Classification 67
3.4 Classification and Regression with Deep Learning 70
3.4.1 Deep Network Designer 70
3.4.2 Network Example: ResNet18 74
3.4.3 Classification Task 80
3.4.4 Regression Task 82
3.5 Regression with Machine Learning 85
3.5.1 Regression Learner App 85
3.5.2 Model Example: Gaussian Process Regression (GPR) 91
3.5.3 Making Predictions with New Data 93
4. Results 94
4.1 Image Preprocessing 94
4.2 Classification Task 97
4.3 Regression Task 100
2

4.3.1 Regression with Deep Learning 100
4.3.2 Regression with Machine Learning 102
4.4 Predictions with 2015 Dataset 104
4.4.1 Wave Height Prediction 104
4.4.2 Water Level Prediction 106
Water Level for Calm Sea State 108
Water Level for Smooth Sea State 110
Water Level for Slight Sea State 112
Water Level for Moderate Sea State 114
4.5 Predictions with 2016 Dataset 116
4.5.1 Wave Height Prediction 116
4.5.2 Water Level Prediction 118
Water Level for Calm Sea State 120
Water Level for Smooth Sea State 122
Water Level for Slight Sea State 124
Water Level for Moderate Sea State 126
5. Discussion 128
5.1 Image Preprocessing 128
5.2 Classification Problem 134
5.3 Regression Problem 137
5.3.1 Regression with Deep Learning 137
5.3.2 Regression with Machine Learning 138
6. Conclusion 142
References 145
Scientific Publications 145
Weblinks 149
Pictorial Sources 157
Appendices 161
3

List of Figures
Figure Title
Page
Fig. 1: Webcam image from Norderney, taken on 8th August 2015 at 10:10 UTC+2.
14
Fig. 2: Beach on Wangerooge aer hurricane Zeynep.
15
Fig. 3: East Frisian Islands offshore of the German North Sea coast.
17
Fig. 4: Groyne field before the nourishment, 18 June 2019 at 9:00 UTC+2.
18
Fig. 5: Groyne field during the nourishment, 23 July 2019 at 9:00 UTC+2.
19
Fig. 6: Groyne field aer the nourishment, 27 July 2019 at 9:00 UTC+2.
19
Fig. 7: Groyne field with placements of combined (1992) and conventional (1989)
nourishments on Norderney.
20
Fig. 8: Webcam position on Norderney.
21
Fig. 9: View of the groyne field from the webcam, 18 July 2015 at 9:00 UTC+2.
22
Fig. 10: Webcam image with the analysis area of the sky framed in red (Reuter,
2022).
24
Fig. 11: Transformation stages of the cropped image of the groyne.
26
Fig. 12: Images from 4th May 2016 and 4th June 2016, and their superimposed
image.
29
Fig. 13: Collage of water lines from the 8th to the 14th of January 2015.
30
Fig. 14: Tide curves of measured and calculated water levels.
30
Fig. 15: Scatter plots for water levels and wave heights.
31
Figure showing ML and DL as parts of AI.
32
Fig. 16: Original and over-segmented image.
36
Fig. 17: Original image and transformed image with ROI.
38
Fig. 18: Images from Farson river camera.
39
Fig. 19: Images taken by an ASV at different sea state levels.
40
Fig. 20: Camera angle of vision with positions of the camera, the wave buoy, and
the water gauge.
42
4

Fig. 21: Image taken by the Norderney webcam on 20th August 2015 at 08:50
UTC+2.
43
Fig. 22: Webcam snapshot from the 18th February 2012 at 12:30 UTC+1.
44
Fig. 23 Webcam snapshot from the 15th September 2017 at 12:10 UTC+2.
45
Fig. 24: Measured vs linearly interpolated wave heights from 2015.
46
Fig. 25: Measured vs linearly interpolated wave heights from 2016.
47
Fig. 26: Tide gauge Norderney-Riffgat.
48
Fig. 27: Water levels from 2015, normalised to the 10-minute resolution of the
image data.
49
Fig. 28: Water levels from 2016, normalised to the 10-minute resolution of the
image data.
49
Figure showing the main preprocessing workflow.
50
Fig. 29: Webcam snapshot with the groyne framed in red.
53
Fig. 30: Webcam image with ROIs for brightness detection.
54
Fig. 31: ROIs for fog and snow detection.
55
Fig. 32: Screenshot of the linking table.
56
Fig. 33: Webcam image from 1. January 2015 at 13:30 UTC+1 and the image
section cropped to the groyne.
57
Fig. 34: Plotted GLCMs for the full-size and the cropped image.
58
Figure showing the main regression workflow.
60
Figure showing the workflow for feature extraction.
63
Figure showing the workflow for water level prediction.
64
Fig. 35: Different image types used for the classification task.
68
Figure showing an average image.
69
Figure showing a difference image.
69
Fig. 36: Main window and model selection menu of the Deep Network Designer.
70
Fig. 37: Active main window of the Deep Network Designer.
71
Fig. 38: Deep Learning Network Analyzer.
72
5

Fig. 39: Import Image Data window of the Deep Network Designer.
73
Fig. 40: Example root-to-leaf branches for the mammal and vehicle subtrees
74
Figure showing the input block of the ResNet18.
75
Figure showing an identity block of the ResNet18.
76
Figure showing an addition and thresholding block of the ResNet18.
77
Figure showing a convolutional block of the ResNet18.
77
Figure showing the classification block of the ResNet18.
78
Fig. 41: ResNet18 structure, simplified block version.
79
Fig. 42: Relative prediction speeds and accuracies of different pretrained
networks.
81
Fig. 43: Different image types used for the classification task.
81
Fig. 44: Layer information for the linear model.
82
Fig. 45: Network architecture for a model with two inputs.
83
Fig. 46: Layer information for the model with two inputs.
84
Fig. 47: Layer graphs for the linear model and the model with two inputs.
84
Fig. 48: “New Session from Workspace” window of the Regression Learner app.
85
Fig. 49: Exemplary response plot.
89
Fig. 50: Exemplary plot for predicted vs true responses.
90
Fig. 51: Exemplary plot of residuals for each observation.
90
Fig. 52: Exemplary overview of the night images with outliers.
94
Fig. 53: Exemplary overview of the twilight images with outliers.
95
Fig. 54: Exemplary overview of the cropped daylight images with outliers.
95
Fig. 55: Exemplary overview of the bad weather images with outliers.
96
Fig. 56: Training plot for classification with normal images and 32 classes.
98
Fig. 57: Training plot for classification with normal images and four classes.
98
Fig. 58: Training plot for classification with edge images and four classes.
99
Fig. 59: Training plot for classification with difference images and 32 classes.
99
6

Fig. 60: RMSE and loss for the training of a ResNet18 for regression.
101
Fig. 61: RMSE and loss for the training of a regression network with two inputs.
101
Fig. 62: Distribution of the measured vs predicted wave heights in 2015.
104
Fig. 63: Scatter plot of predicted vs measured wave heights in 2015.
105
Fig. 64: Measured and predicted wave heights in 2015 plotted over time.
105
Fig. 65: Distribution of the measured vs predicted water levels in 2015.
106
Fig. 66: Measured and predicted water levels in 2015 plotted over time.
107
Fig. 67: Measured and predicted water levels for a two-week period in 2015.
107
Fig. 68: Distribution of predicted vs measured water levels in 2015 during calm
sea conditions.
108
Fig. 69: Scatter plot of predicted vs measured water levels in 2015 during calm
sea conditions.
109
Fig. 70: Predicted and measured water levels in 2015 during calm sea conditions
over time.
109
Fig. 71: Distribution of predicted vs measured water levels in 2015 during smooth
sea conditions.
110
Fig. 72: Scatter plot of predicted vs measured water levels in 2015 during smooth
sea conditions.
111
Fig. 73: Predicted and measured water levels in 2015 during smooth sea
conditions over time.
111
Fig. 74: Distribution of predicted vs measured water levels in 2015 during slight
sea conditions.
112
Fig. 75: Scatter plot of predicted vs measured water levels in 2015 during slight
sea conditions.
113
Fig. 76: Predicted and measured water levels in 2015 during slight sea conditions
over time.
113
Fig. 77: Distribution of predicted vs measured water levels in 2015 during
moderate sea conditions.
114
Fig. 78: Scatter plot of predicted vs measured water levels in 2015 during
moderate sea conditions.
115
Fig. 79: Predicted and measured water levels in 2015 during moderate sea
conditions over time.
115
7

Fig. 80: Distribution of the measured vs predicted wave heights in 2016.
116
Fig. 81: Scatter plot of predicted vs measured wave heights in 2016.
117
Fig. 82: Measured and predicted wave heights in 2016 plotted over time.
117
Fig. 83: Distribution of the measured vs predicted water levels in 2016.
118
Fig. 84: Measured and predicted water levels in 2016 plotted over time.
119
Fig. 85: Measured and predicted water levels for a two-week period in 2016.
119
Fig. 86: Distribution of predicted vs measured water levels in 2016 during calm
sea conditions.
120
Fig. 87: Scatter plot of predicted vs measured water levels in 2016 during calm
sea conditions.
121
Fig. 88: Predicted and measured water levels in 2016 during calm sea conditions
over time.
121
Fig. 89: Distribution of predicted vs measured water levels in 2016 during smooth
sea conditions.
122
Fig. 90: Scatter plot of predicted vs measured water levels in 2016 during smooth
sea conditions.
123
Fig. 91: Predicted and measured water levels in 2016 during smooth sea
conditions over time.
123
Fig. 92: Distribution of predicted vs measured water levels in 2016 during slight
sea conditions.
124
Fig. 93: Scatter plot of predicted vs measured water levels in 2016 during slight
sea conditions.
125
Fig. 94: Predicted and measured water levels in 2016 during slight sea conditions
over time.
125
Fig. 95: Distribution of predicted vs measured water levels in 2016 during
moderate sea conditions.
126
Fig. 96: Scatter plot of predicted vs measured water levels in 2016 during
moderate sea conditions.
127
Fig. 97: Predicted and measured water levels in 2016 during moderate sea
conditions over time.
127
Fig. 98: Daylight image falsely sorted into the twilight category, taken on 4th
February 2015, 10:30 UTC+1.
129
8

Fig. 99: Daylight image with cloudy skies, falsely sorted into the twilight category,
taken on 15th June 2015, 10:00 UTC+2.
130
Fig. 100: Night image wrongly classified as daylight image and cropped, taken on
27th January 2015 at 18:00 UTC+1.
132
Fig. 101: Original and low-light enhanced night image.
132
Fig. 102: Original and low-light enhanced cropped night image.
133
Fig. 103: Two difference images taken during different conditions.
135
Fig. 104: Two edge images taken during different conditions.
135
Fig. 105: RGB images which the edge images in Fig. 104 were derived from.
136
Fig. 106: Scatter plot for water levels from 2015 and 2016.
141
Fig. 107: Scatter plots for water levels from 2015 and 2016.
141
9

List of Tables
Title
Page
Tbl. 1: Thresholds for Daylight Detection
54
Tbl. 2: GLCM of the full-size webcam image.
57
Tbl. 3: GLCM of the cropped webcam image.
58
Tbl. 4: Comparison of the GLCM properties.
59
Tbl. 5: RMSE for regression models without test data.
102
Tbl. 6: RMSEs and models for training on 28 feature columns.
103
10

List of Abbreviations
Abbreviation
Meaning
AI
Artificial Intelligence
ANN
Artificial Neural Network
ASV
Autonomous Surface Vehicle
BN
Bayesian Network
CF
Covariance Function
CNN
Convolutional Neural Network
CET, CEST
Central European (Summer) Time
CPT
Conditional Probability Table
CSV
Comma Separated Values
DirP
Peak Direction
DL
Deep Learning
DRR
Disaster Risk Reduction
DWD
Deutscher Wetterdienst (German Meteorological Service)
ETD
Ebb-Tidal Delta
FCN
Fully Convolutional Network
FoV
Field of View
GLCM
Gray-Level Co-occurrence Matrix
GP, GPR
Gaussian Process, Gaussian Process Regression
GPU
Graphics Processing Unit
HIS
HIstory of Spectral parameters
KAR
Kernel Approximation Regression
k-NN
k-nearest neighbours
LR
Linear Regression
LSTM
Long Short-Term Memory
11

ML
Machine Learning
MLP
Multilayer Perceptron
MRMR
Minimum Redundancy Maximum Relevance
N/A
Not Available
NLWKN
Niedersächsischer Landesbetrieb für Wasserwirtscha, Küsten- und
Naturschutz (Lower Saxon State Department for Waterway, Coastal
and Nature Conservation)
OTUS
Largest Inter-Class Variance, Otsuʼs Method
PCA
Principal Component Analysis
QP
Godaʼs peakedness parameter
R2
Determination Coefficient R-Squared
ReLU
Rectified Linear Unit
RGB
Red-Green-Blue (Colour Space)
RMSE
Root Mean Square Error
RNN
Recurrent Neural Network
ROI
Region of Interest
SGDM
Stochastic Gradient Descent with Momentum
SPR
Source-Pathway-Receptor
SS
Significant Steepness
SSCQ
Sea State Characteristic Quantity
SSVM
Structured Support Vector Machine
SWH, HS
Significant Wave Height
TP
Peak Period
UTC
Coordinated Universal Time
WSA
Waterways and Shipping Administration
12

1. Introduction
Climate change is real, and the Earth is now about 1.1°C warmer than in the late 1800s; the
last decade (2011-2020) was the warmest on record [1]. Among its many effects are the
melting of polar ice caps and increasing sea levels [2]. Since 1880, the global mean sea level
has risen by 21–24 centimetres, and there are predictions that the sea level might increase at
least 0.3 and up to 2.2 metres from 2000 to 2100 [3]. On a planet where 70.8% of its surface is
covered by water, and two-thirds of its population lives within 100 km of the coast (Kunz,
1991), rising sea levels pose an immense threat to the safety of the people inhabiting the
coastal catchment areas.
Monitoring changes to the coast and management of coastal defence measures is therefore
vital to ensure the safety of the people living in coastal areas. However, gathering and
evaluating data reflecting changes, e.g. in beach width or the water level, is oen
time-consuming and expensive. In the past years, artificial intelligence (AI) and related
approaches such as machine learning (ML) or deep learning (DL) have become a cornerstone
of data science. These approaches have been applied in coastal, flood, and water
management, especially to detect water levels. For example, Hies et al. (2012) determined
the water depth in an open storm-water drainage canal in Singapore for watermanagement
purposes using a pressure gauge and ML approaches. Guo et al. (2020) used a similar
approach that they applied to a scaled water gauge. Vandaele et al. (2021) used deep transfer
learning for water segmentation and water level prediction from river camera images, while
Chaudhary et al. (2020) created flood maps for water level prediction using smartphone
images and a DL approach. Besides the water level, other parameters relevant to coastal
engineering can be predicted with ML and DL approaches as well. Hoonhout et al. (2015)
determine the water line and beach width by applying image segmentation techniques to
webcam images. Wang et al. (2013) performed sea state detection using images and a
grey-level co-occurrence matrix (GLCM).
13

What is noticeable in the aforementioned papers is that all of the presented approaches use
either webcam or smartphone images. The developments in camera and lens technology,
especially in combination with the internet as a data-sharing platform, have increased the
volume of high-quality and continuous data available for analysis. Webcams, especially
those stationed at the coast, provide a continuous series of data, even of extreme events
where it would be too dangerous to send people out for measurements. On Norderney, a
webcam has been in use since 2012, and, even though the camera and angle of view were
changed at some points, it has been providing an almost continuous series of high-resolution
coastal images since (Fig. 1). The beach, groyne, and ocean within these pictures can be used
to analyse various coastal parameters, for example, water level, sea state or wave height,
waterline position and more.
Fig. 1: Webcam image from Norderney, taken on 8th August 2015 at 10:10 UTC+2.
Norderney, as one of the East Frisian barrier islands fronting the Lower-Saxonian coast, plays
an integral role in protecting the main coastline from the effects of wind and waves. However,
the wave energy and storm event frequency have increased as a result of climate change and
extreme weather conditions becoming more common. In February 2022, three hurricanes hit
the German coast: Ylenia (16th + 17th Feb.), Zeynep (18th + 19th Feb.), and Antonia (20th +
14

21st Feb.) [4]. These hurricanes damaged the beaches of the islands, which are part of the
coastal protection system and are used to push the surf zone seawards to protect the dike
base. On the coast of Wangerooge, for example, the storm caused massive damage to the
beach (Fig. 2), eroding around 90% of the sand [5].
Fig. 2: Beach on Wangerooge aer hurricane Zeynep.
These effects of storm surges and hurricanes increase the already naturally occurring erosion
due to the tidal current and prevailing wind conditions (Niemeyer, 1995), which negatively
impacts the barrier islands, making more intensive coastal protection measures necessary. A
common method of beach protection at the barrier islands is nourishment with large sand
volumes (Kunz, 1991), especially aer erosion induced by storm surges. However, it is
necessary to evaluate how long these measures last and if they have a beneficial effect on the
protection of the coast; for this purpose, parameters such as beach width and waterline
position could be analysed to assess the performance of the measure. Webcams could
capture these parameters visually in their pictures, and AI-based approaches could be used
to combine the image parameters with measured data to create automatised routines for the
detection of coastal parameters from images.
15

It is therefore the aim of this thesis to develop an AI-based approach for the prediction of
coastal parameters, such as water level and wave height, from webcam images. The model
needs to be able to combine images and measurements (e.g. from wave buoys or tide
gauges) to gain knowledge about which features within an image correspond to certain
parameters and vice versa. The approach should be general enough that new image data
without matching measurements produces equally good prediction results as the data with
matching measurements. Errors should be low enough so that the approach can be used for
real coastal applications. Finally, the routines need to include pre-processing steps and the
option to retrain the model on new data without having to redo the entire model-building
procedure.
Within this thesis, chapter 2 presents information about the study site, the preliminary work,
and the state of the art. Chapter 3 describes the data, how it was pre-processed, and how it
was used for classification and regression tasks with both machine and deep learning.
Moreover, it provides insights into two examples, the ResNet18 deep learning and the GPR
machine learning models. Chapter 4 presents the results of the pre-processing, classification,
and regression, while chapter 5 discusses these results. The conclusion in chapter 6
summarises the findings of this thesis and offers an outlook for potential future research and
applications.
16

2. Literature Review
2.1 Study Site
Norderney is one of the East Frisian Barrier Islands which are located offshore of the Lower
Saxonian part of the German North Sea coast (Fig. 3). East, the Wichter Ee gat separates
Norderney from its neighbouring island Baltrum; west, the broader Norderneyer Seegatt
separates Norderney from Juist. Norderney is located about 3 km further north than Juist
due to the gat Dovetief, which is up to 20 m deep. This offset makes the island more
vulnerable to swell from the West and North-West directions (Mittelstaedt, 2003).
Fig. 3: East Frisian Islands offshore of the German North Sea coast.
The sediment transport by the tidal current from the northern and western direction in the
German Bight causes Norderney, as well as the other Eastern Frisian Islands, to dri away
eastward (Niemeyer, 1995). Between 1650 and 1960, sedimentation caused by the
continuous current resulted in an elongation of the eastern end of Norderney of roughly 6 km
[6]. The East Frisian Islands, unlike the North Frisian Islands, are not part of the continental
shelf and were formed during the Pleistocene (about 2,580,000 to 11,700 years ago) and
Holocene (current geological epoch). Their foundation consists of marine sediments and
(sand) dunes. At the west end of Norderney, its foundation consists of different layers of clay,
marine clay, and loam (Schlütz et al., 2021).
17

This rather “so” Pleistocene shelf in combination with the strong tidal currents made
coastal management procedures necessary. Starting from 1858, the first revetment on the
German North Sea coast was constructed on Noderney (Behre & van Lengen, 1995; Kunz,
1991). The revetment consists of a combined system of shore-parallel structures and groynes.
While strong tidal currents were kept away from the core of the island by the groynes and
further dune erosion could be prevented by the shore-parallel structures, both failed to
prevent beach erosion (Kunz, 1991). The Arbeitsgruppe Norderney 1952 suggested restoration
of the beach aer reviewing the classic coastal protection approach.
Thus, the first beach nourishment on Norderney was carried out in 1951 (Kunz, 1991) to shi
the surf zone further seaward to protect the bank protection structures on the West end of
the island. Since 1951/1952, the nourishments have been repeated regularly every five to ten
years. The most recent (and 13th overall) nourishment was conducted in July 2019, with a
sand volume of 200,000 cubic metres taken from the Robbenplate, a sand bar northwest of
Bremerhaven [7]. The goal was to increase the beach level on a 1.8 km stretch for a total of
eleven groyne fields between groynes F and G1 (see Fig. 4 - 6 for a before, during and aer
view, and Fig. 7 for placements on the coast), with the new level being up to 2 m higher than
before the nourishment [8].
Fig. 4: Groyne field before the nourishment, 18 June 2019 at 9:00 UTC+2.
18

Fig. 5: Groyne field during the nourishment, 23 July 2019 at 9:00 UTC+2.
Fig. 6: Groyne field aer the nourishment, 27 July 2019 at 9:00 UTC+2.
19

Beach nourishments are an established method of coastal protection as the broadening of
the beach pushes the surf zone further seaward, which in turn reduces the risk of a scarp
forming close to coast safety-relevant structures, such as revetments and embankments.
Besides the conventional nourishments, combined nourishments, where both the beach and
the shoreface are replenished, have been carried out on Norderney as well (Niemeyer et al.,
1997; Fig. 7).
Fig. 7: Groyne field with placements of combined (1992) and conventional (1989)
nourishments on Norderney (Niemeyer et al., 1997).
Measures such as traditional (Kunz, 1991)or combined (Niemeyer et al., 1997) nourishments,
as well as changes to the dune revetment (Thorenz & Blum, 2011), need to be evaluated in
regards to their ability to prevent (or at least lessen) beach erosion and protect the
foundation of the dykes. In coastal engineering, parameters that are helpful to evaluate the
performance of coastal management procedures are the water level, wave height, surf zone
width, and the rate of sedimentation vs erosion.
While wave buoys and tide gauges can measure changes in the wave field and the effect of
tidal currents on the beach, they cannot measure erosion. Even though the assessment of
sedimentation and erosion can be done with bathymetric methods such as Lidar, they are
oen too costly to be executed over longer periods. However, developments in the field of
20

camera technology make it possible to gather long-term series of high-quality image data at
low procurement and manpower costs. Additionally, the developments in the field of
artificial intelligence and AI-based image analysis (see 2.3 Machine and Deep Learning) allow
for faster assessments once a suitable model has been established.
In 2012, a webcam - made by the manufacturers Keuschnig and Radlherr, and commissioned
by the Norderney Zimmerservice - was installed at the western end (Fig. 8). Its angle of view
looks onto a section of the groyne field that serves as part of the breakwaters which were
installed to protect the coast from erosion caused by tidal currents and waves. Given that the
camera has a fixed angle which has remained mostly unchanged since its installation(see 3.1
Webcam Technology for details), the produced snapshot images are interesting from a
coastal engineering and management perspective.
Fig. 8: Webcam position on Norderney.
The snapshot images (e.g. Fig. 9) show the boardwalk, the beach, three groynes - the one to
the right being most visible -, as well as the ocean and the sky. These image regions in
combination with data from nearby tide gauges or wave buoys allow for image-based
analysis of coastal parameters, such as wave height, water level, sedimentation, or surf zone
width, making the use of Norderney as a study site an excellent choice.
21

Fig. 9: View of the groyne field from the webcam, 18 July 2015 at 9:00 UTC+2.
22

2.2 Preliminary Work
Background Information
The preliminary work to this masterʼs thesis is the bachelorʼs thesis “Analysis of Webcam
Footage from Norderney Beach” (original German title: Auswertung von Webcam-Bildern
eines Strandabschnitts auf Norderney) by Reuter (2022). The goal of this bachelorʼs thesis was
to calculate water level and wave height and to assess morphological changes of the beach
via analysis of webcam footage, which was taken at the same site this masterʼs thesis focuses
on (see 2.1 Study Site).
Due to obstructions of view and changes of the camera, the timeframe from the 1st of
January 2015 to the 14th of October 2016 was used for the analysis, resulting in a dataset
consisting of 92,000 images (compare 3.1 Webcam Technology). The webcam provides an
image every 10 minutes, the water levels are measured every minute, and the wave height
has a resolution of 30 minutes (compare 3.2 Data Sources). Since the data was linked based
on the image timestamps, it was decided to linearly interpolate short gaps in the wave data
measurements to achieve a 10-minute resolution; for larger gaps in the data, analysis was not
possible. Bathymetric and topographic data for the evaluation of morphological changes on
the beach were not available.
Reuter developed a soware with five main steps. First, data were preprocessed by
combining images with measured wave heights and water levels and then removing images
with low brightness and incomplete data. Second, water levels were calculated from the
image and normalised using the measured data. Third, the height of incoming waves was
calculated by making use of the previously calculated water levels. Next, erosion was
evaluated by comparing images taken during ebb tide at different moments in time. Last, the
results were visualised: for water level and wave height, the calculated results were
presented as scatter plots, while erosion was shown using a comparison image.
23

Preprocessing
The preprocessing started with the creation of a linking table, which joined the image data
with the measured wave heights and water levels. The image and measured data were read
into MATLAB; if there were smaller data gaps, the data were linearly interpolated to achieve a
10-minute resolution. The linking table consists of the path to an image and its timestamp, as
well as the water level and wave height measured at that point. Once the linking table had
been created and saved, the images taken during daylight hours were extracted. The
extraction of these images is necessary, as images taken during night and twilight do not
provide the necessary contrast to perform image-based analysis on them. Using the mean
colour value of an ROI within the sky (Fig. 10), the images were sorted into three categories
Daylight,Twilight, and Night. The ROI does not cover the entire sky to exclude the dark text
and logo at the upper image border and to exclude bypassing images near the horizon, which
could falsify the extraction results.
Fig. 10: Webcam image with the analysis area of the sky framed in red (Reuter, 2022).
The images were transformed from RGB with almost 17 million colours to 8-bit grayscale with
a colour space of 256 colours, which reduced the processing workload of the program, as the
threshold between light and dark could be set to a single value. Images with a colour
24

threshold value below 120 were sorted into the Night category. A threshold value of greater
than or equal to 155 resulted in an image being sorted into the Daylight category. All images
with a colour threshold value below 155 and equal to or greater than 120 were sorted into the
Twilight category. The images were then copied into corresponding tables; the table
containing the daylight images was used for analysis, while the other tables were set aside
for potential later uses.
Water Level Calculation
The preprocessing step provided a table with the image names and corresponding measured
water levels; this table was used to predict the water level between the two groynes on the
images. The images were cropped to show only the area between the groynes and then
rotated to display the water line parallel to the lower image edge. The mean colour values for
the water surface and each image row were calculated to detect the water line; its offset to
the seafront (the lower image edge) was correlated with the measured water levels. However,
since the colour of the beach oen did not differ enough from the colour of the water (e.g.
due to reflection or absorption of light by wet sand), this approach was not pursued any
further.
The approach using mean colour values was altered to focus on an area with a higher colour
contrast; for this purpose, the dark groyne on the right image side was chosen as the ROI.
Images from the Daylight table were rotated 45° counterclockwise to align the groyne
vertically with the lower image edge and cropped to the groyne and some of the area
surrounding it. The image was then transformed multiple times (Fig. 11). First, the RGB image
was transformed to grayscale (Fig. 11a) to reduce the colour space from roughly 17 million to
256 colours, to reduce computation time. An unspecified filter was applied to transform the
grayscale images to black-and-white (Fig. 11b). For each image a threshold value was
extracted from the water colour, and this threshold was used to decide whether a pixel is set
to black or white; the groyne was always set to black while its surrounding was set to white.
In some cases, light reflections could cause a false black colouration of areas within the
image (Fig. 11d). Various other filters were applied to increase the white area around the
groyne to isolate it from other black areas within the images; the black areas outside of the
groyne were deleted aerwards.
25

Fig. 11: Transformation stages of the cropped image of the groyne. (a) Grayscale image, (b)
black-and-white image, (c) image aer application of filters, (d) a wave in the upper image
area is misclassified as part of the groyne (Reuter, 2022).
For the remaining black areas, the geometric centre was calculated (Fig. 11c). Only areas for
which the geometric centre lay within the area of the visible groyne were considered for the
water level calculation. With an increasing water level, the coverage of the groyne with water
increases as well; this fact was used to correlate the covered length to the measured water
level. For this purpose, the distance between the top edge of the image and the first image
row not covered by water was calculated. The number of rows corresponding to this distance
was then correlated with the measured data to allow a direct comparison.
As long as a part of the groyne is still visible, the approach for calculating the water level
works. Once the groyne is fully covered by water (at a water level of around 6.60 m), it can
only be concluded that the water level is high, however, the precise water level can no longer
be calculated.
26

Wave Height Calculation
According to Reuter, the calculation of the wave height was a more difficult task since no
clear image-based feature exists for wave height identification. Therefore, multiple features
and approaches were tested and evaluated.
The first approach was based on the shadow produced by waves. An image region showing
the sea outside of the surf zone was chosen to detect the wave shadows; a darker water
colour was chosen as the indicator for the presence of shadows. Using a histogram of the
grayscale values, a threshold was chosen to differentiate between water with and without
shadows. Individual pixels below this threshold were calculated; the larger the pixel area
below the threshold, the more shadows are visible. While this approach could prove itself
useful for images with a strong contrast between water with and without shadows, it did not
apply to the images from the Norderney webcam, as the contrast was too little. It was
therefore not possible to correlate the summed area of shadows with the measured wave
heights.
Another approach was based on the evaluation of sea foam caused by breaking waves. A
threshold differentiating the white sea foam from the surrounding sea was set based on grey
levels calculated from the sum of the weighted RGB channels (R = 0.299; G = 0.587; B = 0.114).
Diverse lighting conditions caused a strong variation of the threshold between images;
therefore, only images taken between 10:00 and 14:00 were evaluated, as the lighting
conditions were mostly consistent within this timeframe. Pixels above the threshold were
recognized as sea foam, and the total number of these pixels was correlated to the measured
wave height. While this approach provided usable results for very similar lighting conditions,
it was deemed too unstable for a universal application.
Given that the aforementioned brightness-based approaches would result in errors, Reuter
decided to predict the wave heights from the colour values of the surf zone instead. This new
approach consists of six steps. First, images from the Daylight table are read into the
workspace and transformed to grayscale, where seafoam is shown as light while wave
shadows are shown as dark. Then, the images are cropped to show only the surf zone; as the
position of the surf zone varies with the water level, an area sufficiently large to show all surf
27

zone positions was extracted. Aerwards, the extreme values of the greyscale colours were
extracted; instead of using the minima and maxima directly, a tolerance range was applied to
ensure that e.g. people walking through the surf zone do not falsify the results. Next, the
difference between the extreme values was calculated. A high difference shows that bright
(sea foam) and dark pixels (troughs) are present, while a low difference corresponds to a
uniform water surface without waves. Lastly, the calculated differences are correlated to the
measured wave heights.
Assessing Morphological Changes
The morphological changes due to erosion and sedimentation were evaluated for time
intervals between 2013 and 2020, as morphological changes occur on larger time scales than
changes in water level and wave height. Two approaches were used to visualise the erosion:
one superimposed images and highlighted differences through colour, and the other created
a collage of the water lines from different images. Both approaches were applied to three
different intervals: monthly from January 2015 to October 2016 at a water level of 353 cm ± 2
cm, semiannual for May and November from 2013 to 2020 at a water level of 353 cm ± 2 cm,
and event-based for three storm surges between January 2015 and October 2016.
For a water level of 353 cm ± 2cm, the water coverage of the beach is low, allowing for a large
beach width to be assessed. Two images with a temporal distance of one month are
superimposed to highlight the differences between the two images (Fig. 12). In the
superimposed image, the same pixels are grey, while the differences are coloured. For the
groyne, green pixels correspond to erosion, while magenta pixels correspond to
sedimentation. For the beach on the other hand, green corresponds to sedimentation while
magenta corresponds to erosion. Comparisons near the water line and within the surf zone
are problematic, as these vary strongly with the incoming and breaking waves. Additionally,
it is not possible to quantify the movement of sand – a thin layer of sand provides the same
colour information as a thick layer would, given that the lighting conditions are the same.
The constraint of using only images with the same lighting conditions and water levels
results in a high complexity for the image selection.
28

Fig. 12: Images from the 4th of May 2016 (top le), from the 4th of June 2016 (top right),
and their superimposed image (bottom) (Reuter, 2022).
The second approach revolved around creating a collage of cropped regions from multiple
images showing the surf zone and the water line (Fig. 13). Images with the same or a very
similar water level were used, and it was expected that only drastic changes of the water line
position would correspond to morphological changes.
29

Fig. 13: Collage of water lines from the 8th to the 14th of January 2015. The red line
corresponds to the water line, and the middle image shows the storm surge from the 11th of
January 2015 (Reuter, 2022).
Results
For the given image and water level data, the tide curve of the calculated water levels reflects
that of the measured data (Fig. 14) with an accuracy of 73.5% and an RMSE of 38.7 cm. The
coefficient of determination R2is 0.628 for the entire period under review, meaning that
62.8% of the calculated data are within the variance of the measured data. Outliers are
especially prevalent for high measured water levels (Fig. 15, le). 9% of the calculated water
levels lie outside of the first standard deviation.
Fig. 14: Tide curves of measured (blue) and calculated (red) water levels (Reuter, 2022).
30

The calculations of the wave heights did not provide as good results as that of the water
levels. The scatter around the regression line is much higher for the wave height calculations
than for the water level calculations (Fig. 15). Furthermore, R2is 0.238, which means that less
than a quarter of the calculated wave heights are within the variance of the measured values.
However, in 80.5% of the wave heights, the calculated values reflect the general magnitude
of the measured data correctly.
Fig. 15: Scatter plots for water levels (le) and wave heights (right) (Reuter, 2022).
The assessment of morphological changes with the superimposed images showed no
changes for the monthly evaluation. Seasonal morphological changes were detected with
the semiannual evaluation: there was coastal erosion during the winter months and
sedimentation during the summer month. A semiannual evaluation using a collage was not
possible as the camera and its angle of view were changed multiple times between 2013 and
2020. The event-based assessment with superimposed images detected both the
sedimentation caused by a beach nourishment in July 2019 and the erosion caused by storm
surges in the following winter. The collage approach did not detect any changes in the water
line before and aer a storm surge on the 11th of January 2015. The quantitative evaluation
of morphological changes is not feasible using only webcam images.
31

2.3 Machine and Deep Learning
2.3.1 Background Information
Machine learning (ML) and deep learning (DL) are both
subsets of artificial intelligence (AI), with DL being a
subset of ML (see figure to the le, simplified from [9]).
AI itself is a branch of computer science that aims to
replicate or simulate human intelligence in machines
(Hipwell & Alexander, 2022). There are four types of AI,
based on the type and complexity of the task the
system can perform: reactive machines, limited
memory, theory of mind, and self-awareness [10]. In
the scope of this thesis, the type of AI that is being worked with is limited memory, as the
system stores data, features, and past predictions to be able to make new predictions on new
data. For example, the transfer learning that has been applied in this thesis is an ML model
that makes use of limited memory AI [11].
In machine learning, algorithms are trained to identify connections between predictors (e.g.
image features) and responses (e.g. measured data) and to recognize patterns. This learned
information is then used on new, unknown data to make predictions. The algorithms are
capable of writing their code depending on the information they have learned; however, the
user must provide the algorithm with suitable training data. What constitutes suitable datais
dependent on the ML type that is being used; ML is divided into supervised, unsupervised,
semi-supervised (a combination of supervised and unsupervised learning), and
reinforcement learning [12].
For unsupervised learning, the algorithm receives sample data but no target variables.
Dependencies and patterns are learned by the algorithm itself, and there is no “desired”
result provided by the user. Since the learning does not need labelled data, the effort from
the user side is minimal, plus the data can be used in real-time [13]. This type of learning is
used for clustering (e.g. pattern and structure detection using k-means), associations (e.g.
finding correlations using FP-Growth), and reducing dimensionality (e.g. using PCA [principal
32

component analysis] or k-NN [k-nearest neighbours]). However, since the sample data is used
on its own without target variables, this learning approach cannot be used for predictions, as
these need to know true or false outputs beforehand.
Reinforcement learning is a type of machine learning where a system is trained within an
environment and has to make sequential decisions to interact with its environment.
Depending on the cost function, an action is either rewarded or punished [14]. Since this type
of ML is mainly used for robotics and does not work on static data (meaning previously
extracted features and measured target variables), it was not applied – or rather: not
applicable – within the scope of this thesis.
Considering that the goal of this thesis is the prediction of wave heights and water levels
based on image features, the regression models were trained using supervised learning.
During the training stage, the models were provided with sample data (features derived from
webcam snapshots) and target variables (measured wave heights and water levels). Unlike
with unsupervised learning, training data has to be generated by the user by extracting
features from the input data (predictors) and combining them with their corresponding
target variables (responses). Supervised learning is used for regression (numeric output) and
classification problems (discrete output; i.e. probability of a class being true or false) [15].
Deep learning makes use of artificial neural networks (ANN), which have a structure inspired
by human brains and aim to mimic the way neurons communicate with one another. The
three main types of neural networks are multilayer perceptrons (MLP), convolutional neural
networks (CNN), and recurrent neural networks (RNN). MLPs are feedforward networks which
consist of an input layer, one or multiple hidden layers, and an output layer [16]. Since most
real problems are not linear, rather than using perceptrons, MLPs use sigmoid neurons. This
is crucial since the fuzzy logic of sigmoid neurons ensures that small changes in the weights
and biases do not have a large effect on the output, unlike the perceptrons with binary logic,
where a small change might flip the output from e.g. 1 to 0 [17].
CNNs are similar to feedforward networks but apply convolutions to identify patterns from
images, which is why they are oen used for image or pattern recognition and computer
33

vision. RNNs make use of feedback loops and are mainly used on time series data to make
predictions for future events (i.e. stock charts) [16].
The main difference between machine and deep learning is that the latter, by making use of
ANNs, can work with unstructured data (e.g. images, text) by converting them to numerical
values. This extracted information (features) can then be used for e.g. pattern recognition or
predictions, making the approach similar to unsupervised learning. Deep learning is
especially useful for large datasets, as their performance improves with the amount of data
available, rather than reaching saturation the way traditional ML approaches do. However,
the ability to process large amounts of unstructured data and extract features by itself comes
with the downside of high computational requirements. Both the matrix convolutions and
the calculation of the weights of the neurons result in long computation times, making GPUs
necessary, both to speed up the process and to ensure that the hardware can handle the
computational load [18].
Within the scope of this thesis, CNNs (e.g. ResNet18, see 3.4.2 Network Example: ResNet18 for
details) were used for the deep learning approach. The networks were trained for both
classification and regression problems (see 3.4 Classification and Regression with Deep
Learning). The machine learning approach used supervised learning to solve a regression
problem (see 3.5 Regression with Machine Learning).
34

2.3.2 Application in Coastal Engineering
Researching scientific literature for the application of machine and deep learning approaches
to coastal imagery revolved around the following questions: Which approaches exist and
have been used for coastal applications? Were images from webcams or coastal monitoring
stations (e.g. Argus), especially snapshot images, used as inputs? Can these approaches be
used to determine parameters such as water level and wave height? Given the preprocessing
scripts written in MATLAB from the preliminary work of B. Reuter (2022), are any of the
presented approaches applicable in MATLAB? Were any of the approaches applied to barrier
islands, ideally with conditions similar to that of Norderney? A paper that reviewed various
machine learning approaches for coastal engineering purposes is Goldstein et al. (2019).
Within the review paper of Goldstein et al. (2019), three papers were presented that focus on
the application of machine learning approaches to barrier islands. As the study site of this
thesis (see 2.1 Study Site) is a barrier island, the application of ML to other barrier islands
could provide useful cues as to how such an approach could be designed. The papers by
Poelhekke et al. (2016), Plomaritis et al. (2018), and Gutierrez et al. (2015) all apply a
Bayesian network (BN) approach to data from barrier islands to evaluate the morphological
development and changes of the islands over time and especially in response to storm
events. Goldstein et al. (2019) criticise that these papers do not provide access to the codes
used to develop the BNs. While structures, parameters, as well as testing and evaluation
methods are presented, the lack of direct access to the models makes reproduction and
further development of the results difficult if not outright impossible. For example, Gutierrez
et al. (2015) state that the bin widths of conditional probability tables (CPTs) for nodeswere
determined by "subjectively balancing" the need to have enough bins, and that their primary
BN was “designed subjectively”, raising the question as to how these steps could be
reproduced.
Plomaritis et al. (2018) reference the model developed and used by Jäger et al. (2018) for
coastal risk analysis. The presented BN is described in detail and the code is available
open-source [19]; it combines a source-pathway-receptor (SPR) model and disaster risk
reduction (DRR) measures to perform coastal risk analysis and support decision-making in
coastal risk management. The input data for this algorithm consists of hindcast and synthetic
35

extreme event scenarios, information on land use, and vulnerability relationships. Included
within this dataset are water levels, currents, wind conditions, and wave spectra. However,
the model does not use images as an input, and rather than predicting sea states or
numerical values for wave height and water level, hazard probabilities and the effectiveness
of DRR measures are calculated (Jäger et al., 2018).
Since none of the models developed in the aforementioned papers used images as data
input, it is questionable whether they would be suitable for the problem presented in this
thesis, where wave heights and water levels are to be predicted from coastal webcam
imagery. A more suitable approach was found in Hoonhout et al. (2015), where webcam
images were used to determine the water line, among other parameters, by using image
segmentation. Hoonhout et al. used a manually annotated dataset from ArgusNL, consisting
of 192 images from four coastal camera stations along the Dutch coast (2015). These webcams
take snapshot images every 30 minutes; they also store 10-minute mean, variance, min, and
max images, however, these were not used in the paper. The images were over-segmented
and nine classes were manually assigned; these initial classes were then aggregated into five
main classes: water,sand,vegetation,sky, and objects. Oversegmentation was achieved with
the SLIC algorithm to create superpixels (Fig. 16), which are segments of similar pixels, to
boost the number of features for classification. A region-growing algorithm was used to
post-process the over-segmentation to remove scattered superpixels, as these can cause
different regions to be classified as the same region.
Fig. 16: Original (le) and over-segmented (right) image (Hoonhout et al., 2015).
36

Superpixels have variance, patterns, texture, colour, shape, and topological relations, which
were used to generate new features; location, shape, and texture features were used as well.
1727 features from the categories position,intensity,shape, and texture were used by the
classification algorithm, an SSVM (structured support vector machine) that minimises the
regularised cost function. The algorithm was implemented in Python and the performance
was measured by the percentage of correctly predicted classes. An average accuracy of 93.0%
with a standard deviation of 0.7% was achieved for 140,000 test instances. Beach width and
waterline position were determined as well, and variations of the latter could be determined
to O(10m) (Hoonhout et al., 2015).
The above-mentioned papers present approaches which are mainly used to detect the water
line within an image. However, another parameter this thesis focuses on is the water level at
the groyne on the right side of the webcam images from Norderney (e.g. Fig. 9). Using a
combined search with the keywords “machine learning” AND “water level” AND “image
processing”, the goal was to find ML approaches which can be used to identify the water level
from an image. The vast majority of papers matching these terms come from the discipline of
water management, where the ML approach of water level detection is used for drainage
channels or fuel tanks. The methodology of these approaches oen involves a scaled water
gauge within the field of view (FoV) to determine the water level e.g. through image
segmentation or edge detection. While the images from Norderney do not feature a scaled
gauge within their FoV, the groyne in combination with data from a tide gauge at
Norderney-Riffgat can be used to approximate the gauge from the water management
papers. Notable examples are Hies et al. (2012), Vandaele et al. (2021), and Guo et al. (2020).
Hies et al. (2012) determined the water depth in an open storm-water drainage canal in
Singapore for water management purposes. Images from a surveillance camera with a view
of a white pressure gauge were used to extract the water level. These images were
transformed to show the undistorted frontal view onto the gauge, and then the area from the
bottom of the channel to the top of the gauge was extracted as the ROI (Fig. 17). Edge
detection algorithms were applied to find the border between water and gauge, finally,
Hough transformation was applied to find the longest straight line in the edge image, which
corresponds to the water level. The water depth was then interpolated from the water level
37

and previous field measurements. The system can return water depths in real time; during
the study, the camera data had a deviation of 1.1%, while that of the pressure gauge was at
3.9%.
Fig. 17: Original image (le) and transformed image with ROI (right) (Hies et al., 2012).
Guo et al. (2020) use a similar approach, applied to a scaled water gauge. Their preprocessing
steps too consisted of tilt correction, edge detection, and Hough transformation.
Additionally, they used image segmentation using the largest inter-class variance (OTUS, or
Otsu method), which determines the image binarization segmentation threshold, and
difference images. Water levels were classified using sparse representation, resulting in an
error of <0.9cm between calculated and measured water levels.
Instead of using machine learning, Vandaele et al. (2021) used deep transfer learning for
water segmentation and water level prediction from river camera images. Water levels were
estimated from images by using measurements of the heights of particular landmarks or
objects in the field of view of the camera (Fig. 18). Two datasets for semantic segmentation
algorithms were used: COCO-stuff and ADE20k. These were applied on fully convolutional
networks (FCNs); ADE20k was used on an FCN with a ResNet50 encoder and a UpperNet
decoder, while COCO-stuff was used on the DeepLab FCN with a ResNet101 encoder and an
atrous spatial pyramid pooling decoder. Aer the transfer learning phase, experiments were
performed on LAGO and INTCATCH datasets to assess whether sample selection or
fine-tuning provides better results; the latter provided the best results. Overall, the best
results in segmentation could be achieved by training the networks from scratch on the LAGO
38

dataset; while the DeepLab FCN performed better river edge detection, it generally
misclassified more segments in the whole image than the other FCN.
Fig. 18: Images from Farson river camera. Le with landmarks in red dots, and right the
same image with segmented water surface and annotated green landmarks (Vandaele et
al., 2021).
Chaudhary et al. (2020) created flood maps using smartphone images and a VGG16 network
pretrained on the ImageNet dataset. Previously, they had performed flood height prediction
using deep learning and object instance segmentation; building a large, pixel-accurate
annotated dataset had, however, been a considerable effort (Chaudhary et al., 2019). In their
new approach, flood estimation was defined as a per-image regression problem combined
with ranking loss, which reduced the labelling load, as relative ranking is easier to annotate.
The VGG16 network was trained in two parts: in the first, it received known absolute flood
water level for each image, and in the second, the ordering relation. A new dataset
DEEPFLOOD consisting of 8145 ground-level images with water level annotations was created
and then split into two subsets: DF-OBJ, which contains pixel-accurate object instance labels
and flood level annotations per image, and DF-IMG, which contains a single water level
annotation per image. Multiple experiments were conducted on these datasets; a
combination of ranking (trained on DF-IMG) and regression (trained on DF-OBJ) losses
performed best.
39

Fig. 19: Images taken by an ASV at different sea state levels (Wang et al., 2013).
In Wang et al. (2013), sea state detection from images was performed by computing a
grey-level co-occurrence matrix (GLCM) from image texture features and then extracting the
features of the GLCM. A GLCM represents the spatial relationship between pixels in an image,
which reveals texture information, e.g. of homogeneity or recurring patterns. Therefore,
GLCMs and their features play a fundamental role in texture analysis [20]. Wang et al. used
the properties of contrast,correlation,energy, and entropy for their research (2013). Contrast
measures the local variations (difference in pixel values of a reference pixel and its
neighbouring pixel) of the GLCM, correlation measures the joint probability occurrence of the
specified pixel pairs, energy provides the sum of squared elements in the GLCM and indicates
the uniformity within an image, and homogeneity measures the closeness of the distribution
of elements in the GLCM to the GLCM diagonal [21]. The images used by Wang et al. (2013)
were taken by an autonomous surface vehicle (ASV) at different sea state levels (Fig. 19); the
levels ranged from 1 (top row) to 4 (bottom row). The GLCM properties were extracted from
these images to detect the sea state. Analysis of the discernibility between sea states
depending on GLCM features showed that only contrast had differentiated well between the
sea states. It was proposed to improve the discernibility further by computing the SSCQ (sea
state characteristic quantity) as a logarithmic transformation of the contrast. Tests proved
that the SSCQ could indeed improve the discernibility and that the SSCQ as a texture-based
feature was only slightly affected by different light conditions in the images.
40

Jörges et al. (2021) used long short-term memory (LSTM) neural networks to reconstruct and
predict nearshore significant wave height (SWH) to assess the effect of Ebb-Tidal Delta (ETD)
sandbanks on the wave climate offthe Norderney coast. Wave data was taken from thethree
Waverider buoys Coast Ref, Coast I (both north of the ETD sandbanks), and Coast II (located
south of the ETD sandbanks, right in front of the Norderney beach), each with a 30-minute
resolution. Additionally, wind data from the DWD of the Norderney station (10 min
resolution), water level data by the Waterways and Shipping Administration (WSA) for the
gauge level Norderney-Riffgat (1 min resolution), and bathymetries from yearly surveys by
the NLWKN and EasyGSH-DB were used. Two LSTM networks – a standard and a parallel
(P-LSTM) model – were compared to other deep (SL-FFNN) and machine learning (SVR, RF,
MLR) approaches. Tests were done including and excluding the bathymetric data to assess its
influence on the reconstruction and prediction tasks. Using the bathymetric data improved
both tasks; the P-LSTM structure achieved an RMSE of 0.069 m for the reconstruction when
bathymetric data was included.
41

3. Materials and Methods
3.1 Webcam Technology
The webcam located inside the Kaiserhof building in the northwest area of Norderney is a
Canon EOS 1100D reflex camera with an EF-S 18-55 mm / 3.5-5.6 IS II lens [22]. Its viewing
direction is towards the West with a slight tilt northwards (Fig. 20). Within the analysed
period, the focal length is fixed at 18 mm while the aperture and exposure time varies
depending on the lighting conditions.
Fig. 20: Camera angle of vision with positions of the camera, the wave buoy (Boje), and the
water gauge (Pegel) (Reuter, 2022).
The webcam takes a snapshot image (Fig. 21) every ten minutes and uploads it to the
foto-webcam.eu website. The weather data at the top le of the image is provided by the
Deutscher Wetterdienst (DWD) using half-hourly meteorological data from the Norderney
weather station. The logo at the top right is that of Norderney Zimmerservice; the owner and
operator of the webcam. The webcam images have an aspect ratio of 16:9 with an original
size of 4272 x 2600 px and use the RGB colour space without an alpha channel.
42

Fig. 21: Image taken by the Norderney webcam on 20th August 2015 at 08:50 UTC+2.
As seen in Fig. 21, the camera faces the beach at an angle instead of head-on. Thelower third
of the image shows the dike summit and the seafront. Since the image wastaken in summer
during sunny weather, the Kaiserhof building casts a strong shadow onto the grass.
The groyne, which is the designated region of interest (ROI), is framed in red with the label
“OuterRectangle”. It is positioned within the centre third, which shows the beach, the ocean,
and two other groynes. The groynes in the middle le could be used as a control section for
the water level detection, however, due to the angle of view not showing the water cover
ideally, it was ultimately decided against. Given that the rectangular ROI includes parts of the
surf zone, it can be used for wave height prediction as well. The upper third of the image
shows the sky and the aforementioned image annotation. While the lower and upper third do
not include the groyne as the main research focus, they are needed for preprocessing the
images (see section 3.3.1 Preprocessing the Image Data).
While the whole image dataset spans from the 1st of January 2015 to the 14th of October
2016, only the period from the 1st of January to the 31st of December 2015 was used for
model training purposes. Both the 2015 and 2016 subsets were used for predictions. The
2015 data were used for prediction to analyse how much the validation RMSE from training
43

the models might differ from the prediction RMSE. The 2016 dataset from the 1st of January
to the 14th of October 2016 was used for prediction to see how well the model performs on
previously unseen data, where it does not know the true response (measured wave height
and water level). The specific time frame from the 1st of January 2015 to the 14th of October
2016 was chosen to allow for comparability with the results from the preliminary work
(Reuter, 2022). Reuter lists multiple reasons for choosing the specific time frame, and they
will be illustrated here for reasons of comprehensibility.
While the webcam has been taking pictures since the 15th of February 2012 at 12:30 UTC+1,
both the camera and its angle of view have been changed multiple times. As seen in Fig. 22
from 2012, the angle of view differs notably from the one in Fig. 21, which was taken in 2015.
While the angle seen in Fig. 22 was changed on the 23rd of February 2012 and stayed the
same until the camera replacement in 2016, images taken within the first week of operation
do not have an angle comparable to the period of interest. Additionally, many of the images
taken before 2015 contained obstructions in the form of raindrops on the camera lens,
covering the groyne and surf zone, making theimages unsuitable for analysis (Reuter, 2022).
Aerwards, protection against the rain was installed, resulting in only a few images in the
2015-2016 dataset having to be removed due to raindrops covering the view.
Fig. 22: Webcam snapshot from the 18th February 2012 at 12:30 UTC+1.
44

Following the camera change aer construction work in November 2016, the angle of view
was changed once again (see Fig. 23). The images aer the 14th of October 2016 until the
camera change were not used as the view was obstructed by scaffolds. While the new
viewing angle from 2017 onwards might be even more suitable for the deep and machine
learning approaches using only the groyne for water level and wave height prediction, the
comparability with the preliminary work would not be given. Starting from the 22nd of July
2019 (see 2.1 Study Site), the groyne is covered by sand from a beach nourishment, making
the images not suitable for analysis. The sand from the beach nourishment covered the
groyne until early May 2020; aerwards, the groyne is visible enough to be used for analysis
purposes again, and no further long-time obstructions occur.
Next to comparability, availability also played a strong role in the choice of the timeframe.
The foto-webcam.eu server unfortunately only allows images to be downloaded manually
and individually, making it difficult to build a sufficiently large dataset quickly. As the full
dataset from 2015 and 2016 was readily available from Reuter (2022), which is the
preliminary work the performances of the approaches in this thesis are compared to, and it
was chosen as the period of interest.
Fig. 23 Webcam snapshot from the 15th September 2017 at 12:10 UTC+2.
45

3.2 Data Sources
3.2.1 Wave Data
The wave data was provided by the Coastal Research Station of the Lower Saxony Water
Management, Coastal Defence and Nature Conservation Agency (NLWKN) taken from the
Coast II Datawell Directional Waverider MkIII wave buoy located in front of the north-western
beach of Norderney (N 53.715653 E 7.141280) (Reuter, 2022; Jörges et al., 2021). The buoy
records data every 30 minutes; this includes the measurement timestamp, peak period TP,
peak direction DirP, significant wave height HS, Godaʼs peakedness parameter QP, and the
significant steepness SS. The buoyʼs measuring radius for the heave is 20 m in both positive
and negative directions, with a resolution of 0.01 m. For the first three years aer calibration,
the error is less than 0.5% of the measured wave height; aerwards, it increases to less than
1% of the measured values. The range of the direction measurement is 0° to 360° with a
resolution of 1.4°. The heading error is, depending on latitude 0.4° to 2°, but typically 0.5°. A
period of 1.6 to 30 seconds can be measured [23].
Fig. 24: Measured vs linearly interpolated wave heights from 2015.
In the period under review, the measured wave height data contains gaps. While smaller gaps
(e.g. spanning a few hours only) can be resolved by linearly interpolating the data, larger
46

gaps (Fig. 24) would introduce errors and are therefore not considered for the analysis. For
the 2015 dataset, there are three major data gaps: from the 5th of Mayuntil the 4th of June,
from the 5th until the 14th of October, and from the 24th of November until the 8th of
December. The 2016 dataset (Fig. 25) has major gaps from the 4th to the 8th of January, from
the 18th to the 30th of March, from the 14th to the 20th of September, and from the 25th of
September until the 4th of October.
Fig. 25: Measured vs linearly interpolated wave heights from 2016.
The data was not only linearly interpolated to close gaps in data but also to match the
10-minute resolution of the webcam images.
47

3.2.2 Water Level
Fig. 26: Tide gauge Norderney-Riffgat.
The water level data was recorded by the
Norderney-Riffgat tide gauge of the
hydrological information system of the
Federal Waterways and Shipping
Administration (WSV). The black and yellow
mooring post (Fig. 26) is located at the
south-western end of Norderney (53°41'47"
N - 07°09'28" E) [24] and records data every
minute.
The gauge data was provided by the NLWKN
as CSV tables consisting of the records for
one year. For the period under review, from
the 1st of January 2015 until the 14th of
October 2016, the gauge data is mostly
complete.
The gauge data, which was recorded every minute, is then normalised to the timestamps of
the images, which were taken at a 10-minute resolution. Gaps that are visible in the
visualisation of the water level data can be caused either by missing images, as the water
levels are normalised to the image timestamps and images taken during twilight or night
were discarded (see 3.3.1 Preprocessing the Image Data), or by missing gauge data. For the
2015 data, no larger gaps are present (Fig. 27). The 2016 data on the other hand is less
consistent than the 2015 dataset; in early January and mid-August, there are larger gaps
spanning multiple days (Fig. 28).
48

Fig. 27: Water levels from 2015, normalised to the 10-minute resolution of the image data.
Fig. 28: Water levels from 2016, normalised to the 10-minute resolution of the image data.
49

3.3 Processing and Formatting Data
3.3.1 Preprocessing the Image Data
Before using the images for any task or approach, they need to be preprocessed to ensure
the best possible results. The processing steps described in this section focus on extracting
the images taken during daylight hours, creating a linking table between images and other
data, and cropping the images to the region of interest (ROI). The creation of regression data
from the cropped image is described in 3.3.2 Creating Regression Data, while the further
processing of images for classification with deep learning is described in 3.3.3 Processing
Image Data for Classification.
The general procedure (see figure to the le) is as follows:
first, both the folder with the webcam images and the
measured wave height and water level data are loaded into
the workspace. The timestamps of the images are extracted
from the image names. Using the image timestamps, the
measured wave heights and water levels corresponding to
these timestamps are extracted. Gaps in the data are removed
and the data is updated accordingly. Subfolders for the
different lighting conditions are created, and an ROI is defined
from an exemplary image. The images are sorted into
different subfolders depending on the prevalent lighting
conditions; if an image is registered as a Daylight image, it will
be cropped to the bounds of the ROI. Last, a linking table that
joins images, water levels, wave heights, lighting conditions,
and beach visibility is created.
Load Data into Workspace
The first step of the preprocessing workflow is to load the recorded data and imagesinto the
MATLAB workspace to work with them. For this to work correctly, all folders have to be within
the same folder layer, meaning that they have the same parent folder. Here, the folder for the
scripts is called Matlab, while the image folder for the 2015 snapshots is Bilder2015, and the
50

folder containing the measured data is Daten. A list with all images is created from the image
folder; this list is used again later (see Appendix I: Preprocessing with Interpolation).
Extract Timestamps from Images
Next, the timestamps are extracted from the images and saved into the variable zst. All
images have a naming convention similar to norderney-150808-1600-hu with the JPEG
ending. This name contains the timestamp in the format YYMMDD-hhmm, meaning that the
year, month, day, hour, and minute of the time a snapshot was taken can be extracted. The
script loops through all images in the list piclist until all timestamps have been extracted.
Get Wave Heights and Water Levels
Once the timestamps have been extracted from the image names, they can be used to find
the corresponding measured wave heights and water levels. The image timestamps are used
to normalise all data to the 10-minute resolution of the image; normally, the wave heights
are recorded every 30 minutes, while the water levels are recorded every minute (see 3.2
Data Sources).
The script loads the file allewstand.mat into the workspace; this file contains previously
saved water levels wst and their timestamps dattim. If this file has not been created yet, the
script wstand13bis20.m can be used to create this file and its variables from CSV tables of
the gauge data (see 3.2.2 Water Level Data). The preprocessing script then creates a vector of
the size of the image list and assigns it to the variable wstand for the water levels. Using a
loop, all entries wst whose timestamp dattim corresponds to the image timestamp zst are
saved into wstand; therefore, gaps in the image data can result in gaps in the water level
data used for regression.
The wave heights HSare extracted from the file allewelle.mat, which contains previously
saved wave parameters. If this file does not exist yet, the wave parameters – the
measurement timestamp, the peak period TP, the peak direction DirP, the significant wave
height HS, Godaʼs peakedness parameter QP, and the significant steepness SS– can be
extracted from HIS files using the script wellen15bis16.m. Instead of looping through all
data, this part of the preprocessing script finds instances of the image timestamp zst that
51

are found in the vector of wave measurement timestamps dattimwelle. This extracts the
measured wave heights HSfor any image taken at the same time; however, the resulting
vector welleHs can contain gaps. Given that the resolution of the wave height
measurements is 30 minutes and that of the images 10 minutes, there is generally not as
much measured data available as there are images. Using linear interpolation, these small
gaps can be filled. However, the wave data for 2015 and 2016 was unfortunately not
complete, meaning that there are gaps spanning multiple days and sometimes even an
entire month. Filling these gaps with interpolation would result in errors, which is why they
have to be removed.
Remove Gaps and Update the Data
Measured data can contain errors such as gaps or phase shis, which would negatively affect
analysis if le untreated. Visualising the data with plots helps to find larger gaps (see figures
1, 2, 4, and 5 in 3.2 Data Sources). Using MATLAB figures, it is possible to zoom in on the
starting and ending points of the gaps and to get their timestamp by clicking on the dots
corresponding to a measurement. These timestamps can then be used to index into the
image list to remove the gaps from it. If larger gaps were present and removed from the
image list, its dimension is now no longer compatible with that of the timestamp zst, water
level wstand, and wave height welleHs. Therefore, it is necessary to update these variables
so the dimensions of the vectors fit each other again; this is simply done by running the
codes that were previously run to extract the timestamps, water levels, and wave heights
again on the new image list.
Create Subfolders
The next step consists of creating new subfolders for the images sorted by lighting conditions
within the main image folder. First, the path to the main folder isappended by the names of
the new subfolders. Should subfolders already exist and need to be overwritten, they can be
removed by using rmdir. New directories are then made using the mkdir command. These
newly created subfolders are used later for the sorting step of the main workflow.
52

Define ROI from Image
For the task of calculating the wave height and water level from an image, an unchanging
element within the image was chosen as theregion of interest (ROI). For the webcam images
from the 1st January 2015 to the 14th October 2016, this is the groyne on the right image
side. The query of a low water level (wstand < 350 cm) during calm sea conditions (welleHs
< 10 cm) is used to index into the image list to find an image where the groyne is almost
entirely visible (Fig. 29), allowing the user to select the section of the image that shows the
ROI the most clearly. The position of the here-defined ROI is then saved into the variable pos
to be used again in the next step.
Fig. 29: Webcam snapshot with the groyne framed in red.
Sort, Crop, and Save Images
This step sorts the images into different folders based on the prevalent lighting conditions
and crops the images taken during daylight hours to show only the previously defined ROI. A
new variable Light is created which will contain information about the lighting conditions
for each image and will be added to the final linking table. The first image in piclist is
displayed for the masks defined in the loop to work properly.
53

The code then loops through all of the images in piclist and converts them from RGB to
grayscale. Two masks are created for a part of the sea and of the boardwalk; together with a
section of the sky, these are used to detect the lighting conditions (Fig. 30). The dimensions
of the sky ROI were taken from the daylight image detection by Reuter (2022), while those of
the other ROIs were determined from tests with multipleimages. These positions are fixed in
the code and need to be altered if a different angle of view or site is used. The mean of the
grey values is then used to sort the images into different lighting categories (Tbl. 1).
Fig. 30: Webcam image with ROIs for brightness detection. The sky ROI is framed in cyan,
that of the ocean in yellow, and that of the ground in green.
Tbl. 1: Thresholds for Daylight Detection
Light Condition
Sky
Ocean
Ground
Daylight
≥ 150
≥ 65
≥ 30
Twilight
< 150
< 65
< 30
Night
< 100
< 50
< 15
54

In addition to the lighting conditions, the detection of “bad weather” was implemented as
well. The goal is to find images taken during daylight but with bad weather conditions and to
prevent these from being saved into the Daylight folder. Two conditions are aimed to be
detected: fog and snow. Similar to the brightness detection, ROIs are defined to detect fog or
snow (Fig. 31). A polygon covering the sky (without the text and logo) and the beach is used
to detect fog by calculating the range of grey values; if this range is below 200, it is assumed
that fog covers the image. The range was used instead of the mean as fog, similar to Gaussian
blur, reduces details and contrast within an image. Therefore, if the range of values is low,
then this means that contrast is low, and the likeliness of fog is high. Snow is detected by
calculating the mean of the grey values within a polygon that covers the promenade; if the
mean is very high (< 125), then the pixels in this region are rather bright, which points to the
presence of snow, as the grass on the promenade has an otherwise low mean (Tbl. 1). The
positions and thresholds for snow and fog detection were determined empirically from
multiple images and will have to be altered if a different angle of view or site is used.
Fig. 31: ROIs for fog (le) and snow (right) detection.
Only the images that are bright enough to havebeen takenduring daylight hours andno fog
or snow detected are sorted into the Daylight folder. Before being saved into that folder, the
images are cropped to the ROI and have their size reduced by half. For each of the three
lighting conditions Daylight,Twilight, and Night the vector Light received the condition
name as an input.
55

Make Linking Table
The final step of the preprocessing workflow is to create a table that links the images and
their timestamps with measured data and information about the lighting conditions and
beach visibility (Fig. 32). For the beach visibility, a query for the water level is used to find
images where the beach is visible and save this information in the vector Beach. Here, the
beach is considered visible for a water level below 500 cm.
Fig. 32: Screenshot of the linking table.
Motivation for Preprocessing
While the preprocessing of data is time intensive, it has benefits for the tasks that use the
data. Both machine and deep learning work with extracted features – either by receiving
them from the user or by generating them directly from the images. The original images have
a size of 4272 by 2600 px while the cropped and resized images only have a size of 446 by 316
px (Fig 33). Cropping the image to the groyne reduces the influence of other image sections
(e.g. grass or sky) on the image features, which produces more clearly defined features and in
turn more robust results. In addition, the computational load can be reduced by using the
smaller images, as – for this case – they are usually smaller than 25 KB, while the original
images are oen larger than 1 MB.
56

Fig. 33: Webcam image from 1. January 2015 at 13:30 UTC+1 (le) and the image section
cropped to the groyne (right).
Comparing the grey-level co-occurrence matrices (GLCMs) of the original (Tbl. 2) and the
cropped image (Tbl. 3) with one another, it is noticeable that the GLCM of the original image
has no zero elements offside of the main diagonal. If the values of a GLCM are concentrated
along the diagonal, then within the image it is oen the case that two adjacent pixels have
the same value, which means that there are large homogeneous regions present. If the values
of the GLCM are not concentrated along the diagonal, then the image is less homogeneous or
has more contrast [25], as is the case for the original image. The lack of homogeneityof the
original image becomes especially clear when looking at the GLCM represented as a colour
plot (Fig. 34). For the cropped image, only two fields vary strongly from the black
background, while almost the entire diagonal varies from the background for the original
image.
Tbl. 2: GLCM of the full-size webcam image.
57

Tbl. 3: GLCM of the cropped webcam image.
Fig. 34: Plotted GLCMs for the full-size (le) and the cropped image (right).
Comparing the properties of the two GLCMs to one another (Tbl. 4), differences are
noticeable. Contrast, which measures the intensity contrast between a pixel and its
neighbour over the whole image [26], is higher for the original image than for the cropped
image, which means that the latter is more constant. Homogeneity, which measures the
closeness of the distribution of elements in the GLCM to the GLCM diagonal [26], is slightly
higher for the cropped image, meaning that the GLCM is more diagonal and the image has
less contrast (compare Tbl. 2 + 3 and Fig. 34). The correlation, which measures how
correlated a pixel is to its neighbour over the whole image [26], is higher for the original
image. Looking at the original image, there are fewer sections with a stark change in pixel
values from one pixel to its neighbour, while pixels in the cropped image change value
strongly at the borders between sand or water and the groyne. Energy is the sum of squared
elements in the GLCM and returns 1 for a constant or uniform image [26]. As the original
58

image has more contrast and is less homogeneous than the cropped image, it is less uniform
and therefore returns a lower energy value.
While mean, range and variance are not part of the GLCM properties that MATLAB computes
(graycoprops, [26]), comparing them for the original and cropped image is still interesting,
especially since these values are all drastically lower for the cropped image. The lower mean
of the GLCM of the cropped image is a result of the values being lower on average than for the
original image. The lower range means that the difference between the minimum and
maximum values of the cropped image is lower than that of the original image. The lower
variance for the GLCM of the cropped image is a result of the values varying less aroundthe
mean than those of the original imageʼs GLCM. These three values being lower for the
cropped image all point to it being the more homogeneous image, which is the same result
that the graycoprops produced.
Tbl. 4: Comparison of the GLCM properties.
Property
Full-Size Image
Cropped Image
Contrast
0.0665
0.0564
Correlation
0.9887
0.9689
Energy
0.1861
0.4715
Homogeneity
0.9727
0.9737
Mean
1.7351e+05
2.5135e+03
Range
3,014,778
106,368
Variance
1.7974e+23
2.4311e+17
59

3.3.2 Creating Regression Data
While deep learning (DL) approaches can generate features from images by themself,
machine learning (ML) approaches need a priori extracted features as an input (see 2.3.1
Background Information). Here, it will be explained how features are extracted (see Appendix
II: Regression with Interpolation) from images for later use with ML approaches.
The main workflow consists of eleven steps (see figure to the le),
which can contain multiple sub-steps. The routine starts with the
data being loaded into the workspace, and the timestamps being
extracted from the image names. Depending on these
timestamps, the corresponding water levels and wave heights are
extracted. Possible gaps in data are removed and empty arrays for
the features are created. With the gaps removed, thetimestamps
and wave heights are updated accordingly to reflect the removal.
Then, the features are extracted from the images and the empty
arrays are filled with these features. Aerwards, the features are
used by the regression task to predict the wave heights in the
images. Using these predicted wave heights, the water level
prediction is separated into four subcategories for calm,smooth,
slight, and moderate sea. Last, the results are visualised with
different plots and the RMSEs are calculated both for each
prediction as well as for the entire regression task.
Load Data
The routine starts by loading the data into the workspace. For this purpose, the pathsto the
folders with the cropped daylight images (\Bilder2015\Cropped\) and the measured data
(\Daten\) are saved into the variables imgfolder and datafolder, respectively. Both of
these folders, as well as the folder containing the scripts (\Matlab\), are part of the same
parental folder. A list with the paths to all images (piclist) is then created from theimages in
the Cropped subfolder; its size is stored in the variable sz for later use.
60

Extract Timestamps from Images
Next, the timestamps are extracted from the images and saved into the variable zst. All
images have a naming convention similar to norderney-150808-1600-hu with the JPEG
ending. This name contains the timestamp in the format YYMMDD-hhmm, meaning that the
year, month, day, hour, and minute of the time a snapshot was taken can be extracted. The
script loops through all images in the list piclist until all timestamps have been extracted.
Get Wave Heights and Water Levels
Once the timestamps have been extracted from the image names, they can be used to find
the corresponding measured wave heights and water levels. The image timestamps are used
to normalise all data to the 10-minute resolution of the image; normally, the wave heights
are recorded every 30 minutes, while the water levels are recorded every minute (see 3.2
Data Sources). The script loads the file allewstand.mat into the workspace; this file
contains previously saved water levels wst and their timestamps dattim (see 3.3.1
Preprocessing the Image Data). The preprocessing script then creates a vector of the size of
the image list and assigns it to the variable wstand for the water levels. Using a loop, all
entries wst whose timestamp dattim corresponds to the image timestamp zst are saved
into wstand; therefore, gaps in the image data can result in gaps in the water level data used
for the regression task.
The wave heights HSare extracted from the file allewelle.mat, which contains previously
saved wave parameters (see 3.3.1 Preprocessing the Image Data). Instead of looping through
all data, this part of the preprocessing script finds instances of the image timestamp zst that
are found in the vector of wave measurement timestamps dattimwelle. This extracts the
measured wave heights HSfor any image taken at the same time; however, the resulting
vector welleHs can contain gaps. Given that the resolution of the wave height
measurements is 30 minutes and that of the images 10 minutes, there is generally not as
much measured data available as there are images. Using linear interpolation, these small
gaps can be filled. However, the wave data for 2015 and 2016 was unfortunately not
complete, meaning that there are gaps spanning multiple days and sometimes even an
entire month. Filling these gaps with interpolation would result in errors, which is why they
have to be removed.
61

Remove Gaps
Measured data can contain errors such as gaps or phase shis, which would negatively affect
analysis if le untreated. Visualising the data with plots helps to find larger gaps (see figures
24, 25, 27, and 28 in 3.2 Data Sources). Using MATLAB figures, it is possible to zoom in on the
starting and ending points of the gaps and to get their timestamp by clicking on the dots
corresponding to a measurement. These timestamps can then be used to index into the
image list to remove the gaps from it. If larger gaps were present and removed from the
image list, its dimension is now no longer compatible with that of the timestamp zst, water
level wstand, and wave height welleHs. It is necessary to update these variables to fit the
vector dimensions to one another; here, the sizing variable sz is updated for later use.
Make Arrays for Features
Four empty arrays are created to store the image features later. The features that are meant
to be extracted are the variance, mean, range, and properties (contrast, correlation, energy,
and homogeneity) of the grey-level co-occurrence matrix (GLCM). These features return
information on the spatial dependence between pixels and can be used for texture analysis,
which can in turn provide information of e.g. whether waves are present and if the sea
surface is rough or smooth.
Update Timestamps and Wave Heights
As described in Remove Gaps, the variables need to be updated to reflect the new image list
size so that the vector dimensions are compatible. Here, the timestamp variable zst and the
wave height variable welleHs are updated by running the same code with which the initial
values were retrieved. Since the regression script allows the user to choose whether to do the
water level prediction with feature data extracted either from measured or from predicted
wave heights, the waterlevels stored in the variable wstand are not yet updated at this point
in the main workflow.
62

Fill Feature Arrays
Here, the empty arrays created in Make Arrays for Features are filled with feature data. The
code loops through all of the images and extracts the features from them.
The workflow of this sub-task (see figure
on the le) starts by loading an image
into the workspace, which is then
converted from RGB to grayscale using
the rgb2gray command. Aerwards, the
GLCM of the grayscale image is created
using the graycomatrix command.
Multiple properties are extracted from
the GLCM: its mean, range, variance, and
properties contrast,correlation,energy, and homogeneity. The latter four were extracted
using the graycoprops command. As described in 3.3.1 Preprocessing the Image Data,
these features contain information about the texture of an image. Last, the extracted feature
data is saved to two arrays: one for training (tableProps) and the other for prediction
(predData). The difference is that while the prediction data only contains the feature data,
the training data also contains the measured and interpolated wave heights welleHs, as the
ML regression task is based on supervised learning (see 3.5 Regression with Machine
Learning).
Predict Wave Height
The first step of the wave height prediction is to load the previously trained model from the
trainedModel.mat file. This model, as the other models used in the script, was trained in the
native MATLAB Regression Learner App (see 3.5.1 Regression Learner App) and uses Gaussian
process regression (GPR) (see 3.5.2 Model Example: Gaussian Process Regression (GPR)).
Once the model has been loaded into the workspace, it is possible to either start the wave
height prediction immediately using the feature data predData and the line
yfit = trainedModel.predictFcn(predData);
63

or to retrain the model on the previously generated training data tableProps and using the
following code,
[trainedModel, validationRMSE] = trainRegressionModel(tableProps);
which is commented out by default as retraining is only necessary if the angleof view or the
site have changed.
Predict Water Level
The water level is predicted separately from the wave height, as waves on the images can be
interpreted as noise on the water level. In addition, a model that only receives images as
input might not be able to differentiate between two vastly different water levels if the water
surface is similarly rough due to waves, regardless of whether these are tidal or wind-induced
waves. Using the minimum and maximum recorded wave heights, it was determined that five
sea state classes were present: calm (glassy), calm (rippled), smooth, slight, and moderate.
Since the effects of a glassy sea (wave height = 0 cm) and a rippled sea (wave height ≤ 10 cm)
are equally low on coastal protection measures such as revetments, these two sea states
were combined into one, leaving the classes calm,smooth,slight, and moderate.
Therefore, the first step of the
subroutine for the water level
prediction (see figure to the le)
was to set the thresholds for the sea
states. Using the Douglas scale [27],
the following thresholds were set:
wave heights ≤ 10 cm are sorted
into the calm category, wave
heights ≤ 50 cm are sorted into the
smooth category, wave heights ≤
125 cm are sorted into the slight
category, and all wave heights above 125 cm are sorted into the moderate category, which
was possible as no wave heights higher than 250 cm (upper bound for this state) were
measured. If the routine is applied to a case where higher wave heights are present, the
thresholds have to be changed accordingly.
64

Aerwards, the water levels saved in wstand are updated to match the timestamps of the
image list shortened by the removal of gaps (see section Remove Gaps in this chapter). Next,
data for the regression task is created with the updated water levels. This data is created by
combining the feature data stored in predData, the water levels stored in wstand, and the
wave heights. For the wave heights, either the measured data stored in welleHs or the
predicted values stored in yfit can be used. The wave heights are used to index into the
regression data to sort the water levels by sea state. Using the measured wave heights for the
thresholds is useful when creating training data with the water levels, as these values are not
falsified by mispredictions of the regression models. Since the models used in this script had
already been trained and did not need retraining, the predicted wave heights were used to
index into the water levels.
The prediction task starts by loading the trained models into the workspace. They are stored
in files with the naming convention trainedModel*.mat, where *can be Calm,Smooth,
Slight, or Moderate. Similar to the prediction of the wave height, it is possible to either
start the prediction directly or to retrain the models on new training data first. The results of
the predictions are saved into variables yfit_*, where *is the name of the sea state in
lowercase letters.
Visualise Results
The results of the predictions are visualised in multiple ways, always plotting the measured
data against the predicted values. Histograms are used to show the distribution of the data,
displaying how many observations are put into the same range (bin width) for the measured
and predicted data. The wave height and the water levels for smooth and slight seas use a
bin width of 5 cm, as they contain many observations, and opting for a larger bin width (e.g.
50 cm) would not reflect the actual distribution very well. For the water levels of calm and
moderate seas, there were fewer observations available, so the larger bin width of 50 cm
could be used. Depending on the number of observations in general, and per bin, the bin
width has to be changed by the user to fit their desired visualisation of the distribution.
65

Another way of visualising the results is to use a scatter plot that plots measured data against
predicted values and draws the diagonal regression line for the perfect prediction, to assess
visually how well the predicted values reflect the measured data. The observations were also
plotted over time with the predicted values, to assess how much a predicted value differs
from a measured data point. Lastly, a plot over time for all water levels combined is created
to compare the resulting curve of the predicted values to the tide curve from the measured
data, to assess how well the prediction reflects the natural values.
Calculate RMSE
The last step of the main workflow is to calculate the RMSE for the predictions, which is used
to evaluate their quality. The RMSE shows how far a prediction falls off the measured value by
computing its Euclidean distance [28]. In MATLAB R2022a, the RMSE can be computed using
the following code:
rmse = round(sqrt(mean((true(i) - pred(i)).^2)));
Starting from version R2022b, the RMSE can be calculated using the rmse command.
Mathematically, the RMSE can be represented by the following formula, where A is the
measured and F is the predicted value:
𝑅𝑀𝑆𝐸 = 1
𝑛
𝑖=1
𝑛
∑ 𝐴𝑖− 𝐹𝑖
| |
2
Eq. 1
The script loops through all entries in the image list and calculates the RMSE for each
prediction. Once the RMSEs for all individual predicted and measured values have been
computed, possibly existing NaN values are removed using the rmmissing command.
Aerwards, the mean and the standard deviation of all individual RMSEs are computed.
Computing the mean of the individual RMSEs returns the RMSE for the entire dataset, or
better, the fraction that has been analysed. The standard deviation measures the dispersion
of a dataset relative to its mean, or in simpler terms, how much the data is spread around the
mean [29]. The individual RMSEs, the mean RMSE, and the standard deviation of the RMSE
are all calculated in cm for this application.
66

3.3.3 Preprocessing Image Data for Classification
The classification task was done using a deep learning (DL) approach. While DL is capable of
extracting features from images by itself, there is still some preprocessing necessary.
Classification of images with DL needs two inputs: the images themselves and the classes
they belong to. Here, classes were created for sea states and water level groups. Classes can
be created by sorting images into different subfolders and giving these folders names to use
as class names.
Make New Subfolders for Image Classes
Using the linking table created in 3.3.1 Preprocessing the Image Data, the water level
wstand, the wave height welleHs, and the lighting condition Light can be used to index
into the image list piclist to sort images into different categories. For example, using
ind = find(wstand > 400 & wstand < 450 & welleHs <= 10 & Light =
"Day");
sortlist = piclist(ind);
the query of an image taken during daylight hours at a wave height above 400 and below 450
cm with a wave height of less than or equal to 10 cm can be used to index into the image list
piclist, returning all instances for which this query is true and saving it to the variable
sortlist. These images can then be copied into new folders using the code
for i=1:length(sortlist)
copyfile(fullfile(imgFolder, sortlist(i).name), newFolder)
end
where imgFolder is the original folder where all the images are located, and newFolder is
the location for images belonging to a certain class. These folders can be specified in the
following way, where the name of the new folder can be used as the class name:
imgFolder = [wd(1:end-6) 'Bilder 2015\Cropped\'];
newFolder = [wd(1:end-6) 'Bilder 2015\Cropped\400to450calm\'];
67

Create Datastore
The classification task was done in the Deep Network Designer, which expects image data to
be stored in an image datastore (see 3.4.1 Deep Network Designer). Datastores are useful
when individual items of a collection fit into memory but the whole collection does not.
Using the datastore, items from the collection can be loaded into the workspace individually,
saving on memory and computational load. If the data is spread over multiple subfolders
within the same main folder, and their names correspond to names that should be used as
class names for the classification task, an image datastore can be created the following way:
imds = imageDatastore(imgFolder, "IncludeSubfolders", true,
"LabelSource", "foldernames");
Here, imds is the image datastore and imgFolder is the main folder; the name-value
arguments define that subfolders will be included in the data store as well and that their
names will be used as labels (for classes). Such a data store can then be imported into the
Deep Network Designer to classify images based on detected features.
Image Types
Deep learning approaches are capable of extracting features from images without the user
defining these features a priori. However, different types of images contain different
information, and depending on this information, features and results can vary. In the scope
of this thesis, three types of images, all based on the cropped images showing the groyne,
were used: normal RGB images, images showing only the extracted edges, and difference
images (Fig. 35).
Fig. 35: Different image types used for the classification task. Normal RGB image (le),
edge image (centre), and difference image (right).
68

The edge images were created by taking the normal RGB images, converting them to
grayscale using rgb2gray, and then extracting the edges using the edge command with the
approxcanny method, which is an approximate version of the Canny edge detection
algorithm [30]. These images contain information only about the edges present in an image,
such as the borders of the groyne, the waterline, or ripples caused by waves.
The difference images contain information on the differences between one image compared
to an average of all images taken during similar conditions (e.g. wave height, water level).
These types of images can be useful for the automatic generation of features by a DL model,
as they contain only the information that is needed to detect one specific image, rather than
containing additional information that might lead to misclassification.
First, an average image from all images
taken during similar conditions, which are
ideally stored in one folder as described in
Make New Subfolders for Image Classes, is
created using the function code from [31] as
a base. Essentially, an average image (see
figure to the le) is created by adding all
images to one another, and then dividing
the sum through the number of images.
The difference images (see e.g. figure to the
le) are then created by subtracting a
cropped image from the average image,
leaving an image that shows only the
differences between the individual image
and all of the other images taken during
similar conditions.
69

3.4 Classification and Regression with Deep Learning
3.4.1 Deep Network Designer
Both the regression and the classification tasks were done in the Deep Network Designer,
which is part of MATLABʼs Deep Learning toolbox. At the start, in the “Deep Network Designer
Start Page” (Fig. 36), a model to work with has to be chosen. It is possible to choose between
a blank network, a network from the workspace, pretrained image networks, and sequence
networks. The image networks, such as the GoogLeNet or ResNet18 (see 3.4.2 Network
Example: ResNet18), were pretrained on large image datasets (e.g. ImageNet) and are
suitable for transfer learning with image data. Transfer learning allows the use of a pretrained
model on new data, intending to improve the generalisation of the latter by exploiting the
knowledge of the prior [32]. Sequence models are used for classification and regression
problems on sequence and time series data. By choosing a model from the workspace, a
model that had previously been created and exported can be reused. With the blank network
option, a new model can be created from scratch by selecting different layers. In the scope of
this thesis, the pretrained ResNet18 network as well as custom networks were used for
classification and regression tasks on the webcam images (see 3.1 Webcam Technology).
Fig. 36: Main window and model selection menu of the Deep Network Designer.
70

Aer selecting a suitable option for the given task and data, the main window (Fig. 37) will
display different layer options, and, if a premade model has been selected, the model
structure. The model in Fig. 37 is a pretrained ResNet18 network, and the different coloured
blocks in its structure correspond to different layer types. For example, the blue block at the
top is an image input layer that loads 2D images into the model, while the red block at the
bottom is a classification layer which assigns a class to an image aer completing various
matrix calculations and detection mechanisms.
Fig. 37: Active main window of the Deep Network Designer.
If the goal is to use transfer learning with new data on the pretrained network, it is necessary
to change the last three layers. The output size of the fully connected layer has to be changed
either according to the number of classes for classification or to 1 for regression. Additionally,
the somax and the classification layer need to be replaced to match the output size. For
regression tasks, the last layer needs to be a regression layer instead of a classification layer,
the somax layer has to be removed, and the input has to contain predictors (e.g. images)
and responses (e.g. measured data).
As the replacing of layers - or even the creation of a network from scratch - can include errors,
it is useful to use the built-in analyzer (Fig. 38) to determine if the model is plausible,
meaning for example that the output of one layer matches the input of the following layer.
71

Here, the model structure is not displayed as editable blocks but as a layer graph. Layers are
shown as blue dots with the layer name next to them, while the blue lines show the
connections between layers. Branches symbolise the residuals which are included to tackle
the problem of vanishing gradients in very deep neural networks [33]. The table next to the
layer graph contains information about each layer in the network. The first column includes
the name of the layer (here e.g. data [default name would be imageInputLayer]) and a
description of what this layer does in the network (e.g. loading images and applying data
normalisation), while the second columnincludes the layer type(e.g. Image Input). Thethird
column includes the activations of a layer, which consist of the input (size, channels), weight,
and bias. The activations are calculated as shown in Eq. 2.
𝑌 = 𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑖𝑜𝑛( Σ (𝑤𝑒𝑖𝑔ℎ𝑡 * 𝑖𝑛𝑝𝑢𝑡) + 𝑏𝑖𝑎𝑠))
Eq. 2
The output of the activation function will be passed onto the next layer as an input (forward
propagation), and the error from the forward propagation will be used to update the weights
and biases (backpropagation) [34]. These weights and biases are listed in column four as the
learnable properties of each layer.
Fig. 38: Deep Learning Network Analyzer.
72

Once the model has been edited or created and its plausibility has been ensured, data can be
uploaded for training (Fig. 39). In the case of the pretrained ResNet18, the input needs to be
image data. The image width and height can differ from the 224x224 layer input size as
images will be resized during training. The number of channels (here: 3 for RGB) has to match
that of the layer input size, otherwise, the training session will produce an error, and training
will not be started. The data import offers the option of augmentation, which increases a
dataset artificially by modifying the original data (e.g. through rotations or translations) to
create new data points, which can in turn improve the model performance [35]. Additionally,
a specified percentage of the original dataset can be set aside for the validation of the results.
The aim is to prevent overfitting, where a model achieves near-perfect predictions for the
very specific dataset it has been trained on but fails to reproduce these results when
confronted with new data (lack of generalisation) [36].
Fig. 39: Import Image Data window of the Deep Network Designer.
For more detailed information regarding the use of the Deep Network Designer, please refer
to the official documentation by MATLAB, which is linked in [37].
73

3.4.2 Network Example: ResNet18
The ResNet18 is a convolutional neural network that is 18 layers deep and includes residual
layers. The version available in MATLABʼs Deep Network Designer (see 3.4.1 Deep Network
Designer) has been trained on the ImageNet database and can classify images into 1000
object categories [38]. As of 2009, the ImageNet consists of 12 subtrees with 5247 synsets
and 3.2 million images in total. ImageNet uses the same hierarchical structure as WordNet,
where synsets (synonym sets) are concepts that can be described by multiple words or
phrases (Deng et al., 2009). Subtrees start with a broad definition of a concept at the root and
get more precise along the branches until reaching the leaf, where there are no further
synonyms for a concept and no ambiguity regarding the definition is le (Fig. 40).
Fig. 40: Example root-to-leaf branches for the mammal (top row) and vehicle (bottom row)
subtrees (Deng et al., 2009).
Since the ResNet18 is a convolutional neural network (CNN), it consists of two sections, one
for convolution and one for classification, rather than just one section for classification, as is
the case with MLP models (Multilayer Perceptron). During convolution, features are extracted
from the images by running them over multiple kernels with different filter operations (e.g.
edge detection, Gaussian blur). Aerwards, the features will be passed onto fully connected
layers, where the individual features will becombined toclassify the images according to the
classes preset by the user [39].
The ResNet18 receives its name from the residual layers which are part of its structures.
These residual layers were included as very deep networks oen suffer from vanishing
gradients, where the gradient signal quickly reaches zero, which makes their gradient
74

descent prohibitively slow [40]. Vanishing gradients enable deep networks to converge,
revealing a degradation problem where, with increasing network depth, the accuracy
becomes saturated and degrades rapidly aerwards (He et al., 2016). Residual layers are also
known as skip or shortcut connections, as they allow data to bypass the normal flow of the
CNN by skipping one or multiple layers. There are two types of shortcut connections: identity
blocks and convolutional blocks [40]; both are present in the ResNet18 structure.
Diving deeper into the network structure, the 18 layers are blocks of individual layers. There
are five types of blocks present: an input block, the aforementioned identity and
convolutional blocks, addition and thresholding blocks, and a classification block. The layers
within these blocks will be explained in regards to their purpose; the details of individual
layer properties will not be illustrated here but can be accessed via the MATLAB
documentation for deep network layers, linked in [41].
The input block (figure to the le) consists of five different layer
types. The imageInputLayer inputs a 2D image into the network
and applies data normalisation, which reduces the size of the
database by eliminating duplicates, thus making the database
more efficient [42]. For the ResNet18, this layer has an input size of
224x224x3, corresponding to width, height, and the number of
channels (3 for RGB images).
The output of the imageInputLayer is then passed to a
convolution2dLayer, which applies sliding convolutional filters to
the input. Here, a convolutional implementation of the sliding
window algorithm that convolves over the spatial dimensions is
being used. The filter moves along these dimensions and
computes the dot product of the weights and input, and then adds
a bias term [43]. The filters form feature maps, which will then be passed onto the next layer,
which is a batchNormalizationLayer.
75

A batch normalisation layer normalises a mini-batch of data across all observations for each
channel independently [44]. Using this layer type between convolutional layers and
nonlinearities (e.g. ReLU layers) speeds up the training speed and reduces sensitivity to
initialization. As the batch normalisation layers normalise the gradients and activations, the
training problem becomes an optimization problem, which increases the learning speed of
the network significantly (Ioffe & Szegedy, 2015). The output of this layer is then passed to a
reluLayer (ReLU: rectified linear unit), which performs a threshold operation on the input
elements. Values that are less than zero will be set to 0, which can be represented by Eq. 3.
While the values of the input are changed by the reluLayer, the input size will remain
unchanged [45].
𝑓(𝑥) = { 𝑥 , 𝑥 ≥ 0 𝑎𝑛𝑑 0 , 𝑥 < 0
Eq. 3
The final layer of the input block is a maxPooling2dLayer, which performs downsampling
over the spatial dimensions by dividing the input into rectangular pooling regions and
computing their maximum [46]. The output is a pooled feature map that highlights the most
present feature in the map. Downsampling of the feature maps results in lower sensitivity,
making them more robust to changes in the feature position in the image (also referred to as
local translation invariance) [47].
Aer running through the input block, the data is handed to the first
identity block (figure to the le), which consists of two branches. One
is the shortcut connection, which skips over the other layers and
passes the output of the pooling layer straight to the addition layer of
an addition and thresholding block.
The other branch consists of five layers. The first layer of the block is a
2D convolutional layer, which is followed by a batch normalisation
layer, a reluLayer, a second 2D convolutional layer, and a second
batch normalisation layer. The output of the last layer is then
combined with the output of the pooling layer via an addition layer.
76

Identity blocks are standard blocks in residual networks and are used when the activations of
the input and the output have the same dimension [40].
The addition and thresholding block (figure to the right) consists
of two layers, an addition layer and a ReLU layer. While the ReLU
layer performs a thresholding operation, the addition layer simply
adds the two inputs it receives from the main branch and the
shortcut connection to one another. In the structure of the
ResNet18, this type of block always follows aer a residual block,
regardless of whether it is an identity or a convolutional block.
Another type of shortcut connection that
appears in the ResNet18 is the convolutional
block (figure to the le). This block is used
when the dimensions of the input and output
do not match one another [40]. The main
difference, compared to the identity block, is
the presence of a 2D convolutional layer and a
batch normalization layer in the shortcut path.
So, instead of the output from the previous
ReLU layer being passed directly to the next
addition layer, the data in the shortcut path will
be transformed into feature maps and
normalised once before being passed on.
The final block in the ResNet18 structure is the classification block, which assigns classes to
the images the network received as inputs. The first layer of thisblock is a global average 2D
pooling layer (GAP layer), which performs downsampling by computing the mean of the
spatial dimensions. By reducing the total number of parameters in the model, overfitting can
be minimised [48].
77

Aer the convolutional and downsampling layers, the fully connected layer multiplies the
input it receives with a weight matrix and adds a bias vector aerwards. Here, all the features
learned by the previous layers from the image are combined to identify larger patterns across
the image. The last fully connected layer combines the features to classify the images,
flattens the output and encodes spatial data in the channel dimension [49].
The somax layer adds a somax function to its input (Eq. 4). This type of layer usually
follows aer the last fully connected layer in a network [50]
Eq. 4
The final layer of the classification block (figure to the le)
and the pretrained ResNet18 is a classification layer. It
computes the cross-entropy loss for classification and
weighted classification tasks with mutually exclusive classes.
The cross-entropy or log loss is used to measure the
performance of a classification model. Given an output with a
probability value between 1 and 0, a perfect prediction would
result in a log loss of zero. For multiclass classification, the
loss can be calculated as
,
−
𝑐 = 1
𝑀
∑ 𝑦𝑜, 𝑐𝑙𝑜𝑔 (𝑝𝑜,𝑐)
Eq. 5
where Mis the number of classes, yis the binary indicator
for true (1) or false(0), and pis the predicted probability
observation oof class c.
Finally, the individual layers of the network are connected to one another. For the ResNet18,
the blocks of layers follow the structure as shown in Fig. 41.
78

Fig. 41: ResNet18 structure, simplified block version.
79

3.4.3 Classification Task
The classification with deep learning was started by creating multiple subfolders, whose
names were used as classes (see 3.3.3 Preprocessing Image Data for Classification). Classes
were created for different water levels and sea states available in the data, resulting in 43
classes in total. There were ten main water level classes – below and above 350 cm, below
and above 450 cm, above 500 cm, above 550 cm, below and above 625 cm, above 650 cm,
and above 700 cm – as well as five sea state classes for the wave heights: calm glassy,calm
rippled,smooth,slight, and moderate.
Referring to the MATLAB Tutorial “Transfer Learning Using Pretrained Network” [51], a
pretrained GoogLeNet was used to be trained on the webcam images. GoogLeNet is a
convolutional neural network (CNN), 22 layers deep, and was trained on the ImageNet
dataset [52]. For the purpose of transfer learning, where a pretrained network is retrained on
new data, the last three layers of the network – fully connected, somax, and classification
layer – were replaced; the output value of the fully connected layer was set to 43, the total
number of classes. The cropped RGB images of the groyne were used as the image input.
Later, the number of classes was reduced to 32. The classes calm glassy and calm rippled
were combined, as there is no drastic difference in the effect no waves (glassy sea)or waves
below 10 cm (rippled sea) have on coastal protection structures. Furthermore, images where
the weather conditions were not ideal (e.g. snow, fog, sun glint, raindrops) or the groyne was
too obstructed (e.g. covered by sand, strong shadows reducing contrast, completely covered
with water) were removed from consideration.
The network was later changed from the GoogLeNet to the ResNet18, as the latter has a
lower relative prediction time (when using a GPU), which means that it is generally faster, and
has higher accuracy than the GoogLeNet (Fig. 42). In Fig. 42, the area of each marker is
proportional to the size of the network on disk [53]. Next to the RGB images, edge images and
difference images (Fig. 43) were used as inputs for the classification as well (see 3.3.3
Preprocessing Image Data for Classification). The default training parameters were also
changed at times, to assess how e.g. the solver affects the prediction results.
80

Fig. 42: Relative prediction speeds and accuracies of different pretrained networks.
Fig. 43: Different image types used for the classification task. RGB image (le), edge image
(centre), and difference image (right).
81

3.4.4 Regression Task
For the regression task in the Deep Network Designer, different approaches were used. These
approaches consisted of a linear network, a modified ResNet18, and a network with two
inputs. Normally, the input for training regression models is an array or table consisting of at
least one predictor and a response variable. Predictors are features from which the model
can predict a value. During training, these predictors need to have a response assigned to
them as the target variable the model has to predict correctly. The regression with machine
learning was done on 24 feature columns, consisting of the mean, range, and variance of the
grey-level co-occurrence matrix (GLCM).
The first approach involved recreating a linear model from a tutorial on regression with deep
learning published by MATLAB [54]. The input size of the image input layer was changed from
28 x 28 x 1 for a grayscale image to 1 x 24 x 1 for the feature list with 24 columns per row.
Information about these layers is displayed in Fig. 44 below, its layer graph is plotted in the
le image of Fig. 47.
Fig. 44: Layer information for the linear model.
The second approach for the regression task was based around recreating the ResNet18 (see
3.4.2 Network Example: ResNet18) to perform regression rather than classification. For this
purpose, while the layer structure remained the same, all layers had to be replaced. The
input size of the image input layer was changed from 224 x 224 x 3 to 1 x 24 x 1 for the
regression training data. Next, all layers where the property NumFilters (number of filters;
defines the number of neurons within a layer and the number of feature maps it generates as
82

an output) was set to 64 were changed so that NumFilters = 32, which was necessary due to
the changed input size; this change was applied to the convolutional layers. Batch
normalisation and ReLU layers had to be replaced as well, as their parameters had previously
been defined based on the number of filters being 64. The fully connected layer was replaced
and changed to return only one output, which is the numerical value predicted by the model.
Lastly, the somax layer was removed entirely, and the classification layer was replaced by a
regression layer [55].
The last approach was based on a tutorial by MATLAB about training networks on both image
and feature data [56]. Following the network architecture (Fig. 45), the model receives images
and feature data as inputs. The upper branch derives features from the images and combines
them to derive larger patterns in the image; these patterns can then be used for classification
or regression. The lower branch provides the feature data, which consists of previously
extracted features (predictors) and their matching target variables (responses). These two
inputs are combined, and another fully connected layer derives patterns from them. While
the network architecture below finishes with a somax and a classification layer, for the
purpose of regression both of these layers were replaced with a regression layer.
Fig. 45: Network architecture for a model with two inputs.
The image input layer has a size of 313 x 412 x 3, which is the same size andchannels as the
cropped RGB images, and the feature input layer has a size of 1 x 24 x 1, the same as the size
of the regression data. Information about these layers is displayed in Fig. 46 below, while its
layer graph is plotted in the right image of Fig. 47.
83

Fig. 46: Layer information for the model with two inputs.
Fig. 47: Layer graphs for the linear model (le) and the model with two inputs (right).
84

3.5 Regression with Machine Learning
3.5.1 Regression Learner App
The regression task was done in the Regression Learner App, which is part of MATLABʼs
Statistics and Machine Learning Toolbox. At the start, in the main “Regression Learner”
window, it is possible to either start a new session – either by selecting data from the
workspace or importing data from a file – or to open a previous section. When opting to use
regression data from the workspace, a new window “New Session from Workspace” (see Fig.
48) will open; here, the training data can be selected.
Fig. 48: “New Session from Workspace” window of the Regression Learner app.
85

Under the section “Data set”, subsection “Data Set Variable”, the main dataset for the
regression task can be chosen. It ispossible to select between usingthe columns or the rows
as variables; in the scope of this thesis, the default “Use columns as variables” was used, as
the training data had been created with each array column corresponding to a predictor or
response (see 3.3.2 Creating Regression Data).
In the subsection “Response”, the response variable for the dataset can be chosen. It is
possible to choose the response either from a dataset variable and specify which column or
row should be used as the response variable or to import the response variable from the
workspace. The default option “From data set variable” was used in this thesis, and the 29th
column was chosen, as it contains the measured wave heights or water levels (see section
3.3.2 Creating Regression Data). The predictor variables that are to be used for the regression
can be selected under the subsection “Predictors”. By default, when going with a response
from the data set variable, all other columns or rows are used as predictors. For this
regression task, the default was used, however, individual columns can be deselected in case
that it is known in advance that they are not suitable as predictors.
Under the section “Validation”, the validation scheme can be chosen. Validation of the
training data is necessary to protect against overfitting. Overfitting describes the process
where a model becomes overly complex by aiming to match the initial training data points as
closely as possible [57]. This is a problem, as the model will only be able to perform well on
the training dataset but will perform much worse with new data; a lack of generalisation is
introduced by the overfitting. In the Regression Learner App, there are three validation
schemes from which can be chosen: Cross-validation, holdout validation, and resubstitution
validation. Holdout validation sets aside some of the data for the validation and only runs the
validation once. The cross-validation splits the dataset into k subsets of equal size and
computes the validation loss for each of them. For resubstitution, the error is calculated
based on true vs predicted value; it is not suitable to prevent or reduce overfitting [58].
86

The default option is cross-validation, which was used for the regression task of this thesis.
When cross-validation is chosen as the validation scheme, the number of folds can be
specified. The default is 5, which was kept in the scope of this thesis. Finally, it is possible to
choose if a data set should be set aside for testing under the section “Test”. No data is set
aside by default, and aer comparing the performance of models measured by the RMSE (see
4.3.2 Regression with Machine Learning), it was decided against setting aside data for test
purposes.
Training a Regression Model
Once the data, response, predictors, validation scheme, and test data have been chosen, the
session can be started by clicking the “Start Session” button. This action opens the data in
the main window, where the Tree model (Fine Tree) will be chosen by default to visualise the
target variable with a response plot.
There are multiple options available for trainable models. Under “Get Started”, it is possible
to train either all available models or only the models that are fast to train. With “Use
Parallel”, multiple models can be trained at the same time. The options for linear regression
(LR) models are Linear,Interactions Linear,Robust Linear, and Stepwise Linear. For regression
trees, fine, medium, and coarse trees are available. Support vector machines (SVMs) are
available with different kernel functions: linear, quadratic, cubic, and Gaussian; the Gaussian
kernel functions are available as fine, medium, and coarse. Gaussian process regression
(GPR) is available with a Rational Quadratic,Squared Exponential,Matern 5/2, or an
Exponential kernel. The kernel approximation regression (KAR) models are available as SVM
Kernel or Least Squares Kernel Regression – both of these are Gaussian kernel regression
models for nonlinear regression with a large data volume. The ensembles of trees, which
combine multiple regression trees, are available as Bagged (goal: reduce the variance of a
decision tree) or Boosted (goal: create a collection of predictors) [59]. The neural networks
(NN) are available as narrow,medium,wide,bilayered, and trilayered networks. For all of the
model groups, it is possible to load all models to be trained into the model space at once.
Except for the LR and the KAR models, all model classes have an optimizable model
available, which can be used for hyperparameter optimization.
87

There are three main optimizer options: Bayesian optimization, grid search, and random
search. The acquisition functions for the Bayesian optimizer are expected improvement per
second plus,expected improvement,expected improvement plus,expected improvement per
second,lower confidence bound, and probability of improvement. For the grid search, the
number of grid divisions can be specified. It is possible to set a training limit or a maximum
number of iterations for each optimizer.
Besides the optimizer to determine the hyperparameters resulting in the best performance, it
is also possible to do feature selection and enable PCA. Feature selection makes use of
different feature ranking algorithms to determine which features have the most importance
for or effect on the prediction of the response variable. There are three options for which
ranking algorithm should be used on the features: MRMR, FTest, and RReliefF. Aer the
importance scores have been computed by the ranking algorithms, it is possible to select
either how many of the highest-ranked features should be kept or to select individual
features for training. In addition, principal component analysis (PCA), can be used to reduce
the dimensionality of a dataset to increase the simplicity and thereby the training speed of
the model. PCA aims to reduce the number of variables while preserving as much
information as possible [60].
Once the training has been finished, it is possible to test the models either with data
previously split from the training set or with new data. Testing is useful to see how well the
trained model performs when confronted with new data, as models may overfit themselves
to the training data, resulting in a loss of generalisation. Underfitting, where models are too
general to fit the training data, is possible as well, however, overfitting occurs more
frequently.
88

Visualising the Results
Aer training, as well as aer testing, the Regression Learner App provides multiple plots to
visualise the results. The response plot (Fig. 49) displays the response value (y-axis) for each
observation (x-axis). In the plot, true values (the responses or target variables) are
represented by round blue markers, while the predicted values are represented by round
yellow markers.
The plot of the predicted vs actual values (Fig. 50) displays the true values on the x-axis and
the predicted values on the y-axis. The round blue markers represent the individual
observations, and the black diagonal line is the regression line, which expresses the perfect
prediction with zero error. Lastly, the residuals plot (Fig. 51) visualises the offset or error each
predicted value has in comparison to the true value at the same observation (here: record
number). All of these plots aim to visualise how well the prediction performs in direct
comparison with the true data.
Fig. 49: Exemplary response plot.
Blue dots represent the true
responses, while yellow dots
represent the predicted responses.
89

Fig. 50: Exemplary plot for
predicted vs true responses.
Blue dots represent the individual
observations while the black
diagonal equals the regression line
of the perfect prediction.
Fig. 51: Exemplary plot of residuals
for each observation.
The red lines represent the error of
the predicted response for each
observation.
90

3.5.2 Model Example: Gaussian Process Regression (GPR)
Gaussian process regression (GPR) is a form of supervised machine learning created to solve
prediction or regression problems. The Gaussian process (GP) prediction is, by definition, a
Gaussian distribution [61], and can be fully specified by a mean function m(x) and a
covariance function k(x, xʼ) (Rasmussen & Williams, 2005b). The definition of a GP as a
collection of random variables implies a consistency property (Rasmussen & Williams,
2005b), which means that the same prediction procedure can be applied to all data available,
regardless of the volume.
GPR is probabilistic and non-parametric [62]. Here, probabilistic means that, due to the
uncertainty being reduced close to the observations (Rasmussen & Williams, 2005a),
predictions can be made based on how likely their features reflect properties that the
original observations possessed. Non-parametric models do not compress the training data
into a finite-dimensional parameter vector, which means that they have an infinite number of
parameters (Rasmussen & Williams, 2005e). Realistic observations can be represented as y =
f(x) + ε, where f is the underlying signal that the GPR aims to reconstruct to predict the values,
and ε is the contaminating noise that needs to be removed (Rasmussen & Williams, 2005b).
Given that the removal of the noise is equal to smoothing the observations, the GPR can be
seen as a linear smoother (Rasmussen & Williams, 2005e). Both the hyperparameters and the
covariance function, which will be explained in the following paragraphs, have a strong effect
on the GPR and its results [62].
The kernel or covariance function k(x, xʼ) of the GPR relates observations to one another
(Ebden, 2008). The similarity of two inputs x and xʼ is measured, and by making inferences
about the relationship between inputs and outputs, it is determined what makes x and xʼ
similar. In the scope of this thesis, exponential (Eq. 6) and Matern 5/2 (Eq. 7) kernels were
used, which are stationary and nondegenerate (no finite rank). A stationary CF is a function of
x - xʼ, which means that it is invariant to translations of the input space. According to Mercerʼs
theorem, there exists a (possibly infinite) expansion in terms of basis functions for every
positive definite covariance function. CFs have free parameters, the hyperparameters; for
many CFs, their meaning is relatively simple, which is important for understanding the data
(Rasmussen & Williams, 2005d).
91

Eq. 6
Eq. 7
σlis the characteristic length scale (l), σfis the signal standard deviation, and r is the
Euclidean distance between xiand xj[63].
The hyperparameters of the kernel can be defined as θ = {l, σf, σn} (Ebden, 2008). The
parameter lis the length or horizontal scale, which represents the distance to move in input
space before function values change significantly. The effect of observations depends on the
characteristic length scale l and will be negligible for distant observations (Ebden, 2008). If l is
too short, the result is a quickly varying signal with low noise, however, if l is too long, the
result is a slowly varying signal with high noise (Rasmussen & Williams, 2005b). The inverse of
the length scale determines how relevant an input is; this correlation is used e.g in automatic
relevance detection (ARD) (Rasmussen & Williams, 2005d). The output or vertical scale is
given by σ2; where σn2is the noise variance (also known as σε) and σf2is the signal variance.
The signal variance or maximum allowable covariance is high for functions which cover a
broad range on the y-axis (Ebden, 2008). If the noise variance is high, the model will ignore
the data, however, if the noise variance is very low, the data will be taken very literally [63].
92

3.5.3 Making Predictions with New Data
Before making predictions with new data, it is necessary to get the right input format. The
raw image data from the webcam needs to be preprocessed to contain only the ROI (see 3.3.1
Preprocessing the Image Data) and then features need to be extracted from the cropped
images (see 3.3.2 Creating Regression Data). Here, the size of the array with the extracted
feature data is n x 28 x 1, where n is the number of observations (rows), 28 is the number of
feature columns, and 1 is the feature dimension. Aer the feature data for prediction has
been created, the trained regression models can be loaded into the workspace. Both the
model (e.g. trainedModel.mat) and the script their prediction function predictFcn
references (e.g. trainRegressionModel.m) must be within the same folder as the script
that calls them (here: regression_withInterp.mlx). The prediction can then be executed
using the following code:
yfit = trainedModel.predictFcn(predData);
The variable predData contains the aforementioned prediction data, and the variable yfit
will be filled with the predicted values. However, if the new images vary from the original
images, then the values of the newly extracted features will differ from the original training
features, which can result in erroneous predictions. Such a problem can be prevented by
retraining the model on the new data, using the following code:
[trainedModel, validationRMSE] = trainRegressionModel(tableProps);
The training function trainRegressionModel, which is stored in the same script as the
prediction function predictFcn, retrains the model on the dataset tableProps. This training
dataset has the size n x 29 x 1, where the first 28 columns contain the image features, and the
last column contains the response or true value. However, for vastly different image or
prediction data, it might be more sensible to train a completely different model, e.g. by using
the options All or All Quick-to-Train in the Regression Learner App (see 3.5.1 Regression
Learner App).
93

4. Results
4.1 Image Preprocessing
The routine from 3.3.1 Preprocessing the Image Data (section Sort, Crop, and Save Images)
was implemented to sort images into three different lighting conditions: daylight,twilight,
and night. In addition, the daylight images were also sorted by weather conditions: if the
conditions were good, the images would be cropped and saved to the folder Cropped, and if
the conditions were bad (e.g. snow or fog present), they would be sorted into the folder
BadWeather without the cropping.
The detection routine worked best for the detection of night images, where the outliers
consisted mainly of images taken during twilight, and sporadically images taken during the
day but with low lighting conditions (e.g. in winter during snowfall) (Fig. 52). Generally, the
amount of correctly identified night images far outnumbered the falsely classified images.
Fig. 52: Exemplary overview of the night images with outliers.
94

The detection of the twilight images (Fig. 53), on the other hand, performed worse. The folder
for the twilight images contained many images taken either during the night or daylight
hours; roughly estimated, at least half – up to almost two-thirds – of the images were
misclassified.
Fig. 53: Exemplary overview of the twilight images with outliers.
Compared to the detection of twilight images, the detection of daylight images classified
most of the images correctly; some outliers from the night or twilight images existed (Fig. 54).
Fig. 54: Exemplary overview of the cropped daylight images with outliers.
95

The detection of bad weather from daylight images worked well (Fig. 55); hardly any images
taken during good weather conditions and daylight hours were falsely sorted into this folder.
Fig. 55: Exemplary overview of the bad weather images with outliers.
96

4.2 Classification Task
The classification using deep learning was performed as described in 3.4.3 Classification
Task, where different model approaches and different data were used to solve the problem of
assigning images to classes.
The first approach used the original dataset with 43 classes consisting of the cropped RGB
image and applied it on a pretrained GoogLeNet model. This resulted in a validation accuracy
of 38% and a loss greater than 2. A reduction of classes from 43 to 32 resulted in an improved
validation accuracy of 57%. These tests used the default training parameters, except for the
following parameters: the output network was set to return the model with the lowest
validation loss, the mini-batch size was increased from 128 to 256, the validation frequency
was reduced from 50 to 5, and the maximum epochs were reduced from 30 to 10.
Changing the model from the GoogLeNet to a ResNet18 and increasing the momentum from
0.9 to 0.95 resulted in a validation accuracy of 59.9888% and a validation loss of 1.0247,
which was slightly better than the previous result with the GoogLeNet. The same test was
repeated on a PC with a GPU supported by MATLAB (see Appendix III: PC Specifications and
MATLAB® Toolboxes). While the validation accuracy (60.7946) and loss (0.9981) only improved
marginally, the computation time was reduced from over 17 hours to just under one hour.
Increasing the L2-Regularization to 0.01 resulted in a validation accuracy of 62.28%, while
using the rmsprop solver instead of the SGDM solver returned a validation accuracy of
64.45%. Aer cleaning up some falsely sorted data and reusing the same parameters as
before, the validation reached an accuracy of 69.91% (Fig. 56). Using only four sea state
classes (calm,smooth,slight, and moderate) and a mini-batch size of 128, the validation
accuracy reached 86.42% with a loss of 0.3792 (Fig. 57).
97

Fig. 56: Training plot for classification with normal images and 32 classes.
Fig. 57: Training plot for classification with normal images and four classes.
When edge images were used instead of the normal RGB images, the accuracy fell to 72.49%
with a loss of 0.73 for four classes (Fig. 58).
98

Fig. 58: Training plot for classification with edge images and four classes.
The best performance was achieved using 32 classes containing difference images, the
rmsprop solver, and the output network being the one that produces the lowest validation
loss, resulting in a validation accuracy of 98.42% with a loss of 0.0521 (Fig. 59).
Fig. 59: Training plot for classification with difference images and 32 classes.
99

4.3 Regression Task
4.3.1 Regression with Deep Learning
Three approaches were used for the regression with deep learning: a linear model, a
modified ResNet18, and a network with two inputs (see 3.4.4 Regression Task). The linear
model was the first attempted approach, however, it failed due to an error with the data
format. During the time the data format error was being resolved, regression was performed
in the Regression Learner App. When the data format error was solved, instead of trying the
linear model again, which had produced an RMSE so high its value was N/A (not available) on
a test dataset provided by MATLAB, it was decided against testing this approachany further.
Instead, the other two approaches became the focal point of the regression with deep
learning.
Using the modified ResNet18 with the SGDM solver on normal RGB images, an RMSE of
25.319 was achieved. The rmsprop solver performed slightly worse, with an RMSE of 25.387,
and the adam solver performed slightly better, with an RMSE of 25.171 – however, since these
RMSEs were predictions of true values in whole centimetres, they can all be rounded to an
RMSE of 25 cm, meaning that there is no significant difference between these values. While
all of these models were trained for over 1,000 iterations, they all reached RMSE and loss
plateaus aer about 500 iterations (Fig. 60).
The first RMSE produced by the two-input network, which used 24 feature columns and
images as inputs – had a value of 240,829,600 cm (ca. 2.4*10e8 cm). It was trained with the
SGDM solver and a mini-batch size of 64. Other attempts with this network performed
similarly. Using the expanded feature set (28 columns) and the Adam solver, the RMSE was
improved to 19,057 cm (ca. 1.9*10e4 cm) for the final training iteration (Fig. 61).
100

Fig. 60: RMSE and loss for the training of a ResNet18 for regression.
Fig. 61: RMSE and loss for the training of a regression network with two inputs.
101

4.3.2 Regression with Machine Learning
The initial tests in the Regression Learner App focused on training all available models with
features (24 columns) generated from either difference or normal RGB images, for the
following datasets: wave heights, water levels during calm, smooth, slight, or moderate sea,
and all water levels. The lowest RMSEs that were achieved for training models without
setting aside data for tests are recorded in Tbl. 5 below – the values were rounded to the
nearest integer.
Tbl. 5: RMSE [in cm] for regression models without test data.
Dataset
Difference Images
Normal Images
All Wave Heights
–
19
All Water Levels
–
36
– calm sea
–
28
– smooth sea
87
36
– slight sea
68
35
– moderate sea
25
18
There were no runs done on difference images for all wave heights, water levels, and the
water level during calm seas. For the first two, this is the case as by the time thesedatasets
were passed to the models for training, it had already been clear from the results of the other
models that the normal RGB images outperform the difference images. While there is no
RSME for training on difference images for the calm sea state without test data, there are
results for setting aside test data: the RMSE for the difference images was 38 cm higher than
for the RGB images.
The models that performed best were Gaussian Process Regression (GPR) with a Matern 5/2
kernel and medium neural networks. Using feature ranking algorithms to reduce the number
of features, PCA to reduce the dimensionality, and optimization to optimize the
hyperparameter all did not improve the results.
102

Expanding the initial feature data consisting of 24 columns with the properties of the GLCM
(graycoprops;contrast,correlation,energy,homogeneity) improved the performance of the
different regression models slightly (Tbl. 6). Training the models only on the graycoprops
increased the RMSE rather than decreasing it.
Tbl. 6: RMSEs and models for training on 28 feature columns.
Training Data
RMSE [cm]
Best Model
Wave Heights
18
Exponential GPR
Water Level, calm sea
26
Matern 5/2 GPR
Water Level, smooth sea
35
Exponential GPR
Water Level, slight sea
32
Exponential GPR
Water Level, moderate sea
18
Exponential GPR
103

4.4 Predictions with 2015 Dataset
4.4.1 Wave Height Prediction
The wave height prediction reached a mean RMSE of 11 cm for the whole dataset and a
maximum RMSE of 168 cm for an individual prediction. The distribution of the measured vs
predicted wave heights (Fig. 62) shows how accurately the prediction fitted the measured
values. The bin width was set to 5, so one bar equals a wave height range of 5 cm. It is
noticeable that for wave heights below 25 cm and above 100 cm the prediction remained
below the measured values, while the predictions were too high for measured wave heights
between 25 cm and 100 cm.
Fig. 62: Distribution of the measured (blue) vs predicted (orange) wave heights in 2015.
By plotting predicted against measured values and drawing the diagonal of the perfect
prediction through the scatter, it can be assessed how closely the regression predicted the
true values, and if it under- or over-predicted them. In Fig. 63 it is noticeable that for wave
heights below 20 cm, the prediction was higher than the true values, while wave heights
above 100 cm were under-predicted. This result is agreeing with the findings of the histogram
depicting the distribution of the wave heights (Fig. 62).
104

Fig. 63: Scatter plot of predicted vs measured wave heights in 2015.
Plotting the measured and predicted values over time (Fig. 64) shows that while the
regression under-predicted high and over-predicted low wave heights, the prediction
generally fitted the chronological progression of the measured wave heights.
Fig. 64: Measured (blue) and predicted (red) wave heights in 2015 plotted over time.
105

4.4.2 Water Level Prediction
The water level detection reached a mean RMSE of 26 cm for the whole dataset and a
maximum RMSE of 216 cm for a single observation. The distribution of the measured vs
predicted water levels (Fig. 65) shows how accurately the prediction fitted the measured
values. The bin width was set to 5, so one bar equals a wave height range of 5 cm. It is
noticeable that water levels below 390 cm and above 600 cm were under-predicted by the
regression, while especially the range from 390 cm to 500 cm was over-predicted. For the
range from 500 to 600 cm, both over- and under-prediction occurred.
Fig. 65: Distribution of the measured (blue) vs predicted (orange) water levels in 2015.
Plotting the predicted and measured values over time (Fig. 66) confirms the findings from the
histogram (Fig. 65) that low water levels were over-predicted while high water levels were
under-predicted by the regression model. Fig. 66 shows the general placement of predicted
values against the measured values, but it does not offer information on how closely the
predicted tide curve follows the measured one. Therefore, a period of two weeks was
randomly chosen to assess the course of the tide curve (Fig. 67). It is noticeable that the
course of the predicted tide curve follows that of the measured curve quite closely, however,
it does not match the peaks and valleys (high- and low-tide) of the measured curve as well as
it does with the general course of the curve (mid-tide).
106

While this section looked at the results for all water levels, the following sections will take a
closer look at the water levels sorted by sea state, which were used to make the predictions.
Fig. 66: Measured (blue) and predicted (red) water levels in 2015 plotted over time.
Fig. 67: Measured (blue) and predicted (red) water levels for a two-week period in 2015.
107

Water Level for Calm Sea State
The water level detection for calm seas reached a mean RMSE of zero for the whole dataset
and a maximum RMSE of 127 cm for a single observation. The distribution of the predicted vs
measured water levels for calm sea conditions (Fig. 68) has a bin width of 50, meaning that
one bar has a water level range of 50 cm. While the water level between 500 and 550 cm
seems to have been predicted perfectly, the other ranges were either over- or
under-predicted by the regression model.
Fig. 68: Distribution of predicted (orange) vs measured (blue) water levels in 2015 during
calm sea conditions.
The regression plot (Fig. 69) shows that while there is scatter for the water levels below 500
cm and above 550 cm, most of the predictions for the range of 500 to 550 cm were perfect.
This reflects the distribution of the histogram in Fig. 68. In the plot over time (Fig. 70), where
the predicted and measured values are plotted at each measurement time point, the high
errors of some individual predictions are especially visible. However, it also shows that most
of the predictions for observations in early October during low tide were fairly accurate.
108

Fig. 69: Scatter plot of predicted vs measured water levels in 2015 during calm sea
conditions.
Fig. 70: Predicted (red) and measured (blue) water levels in 2015 during calm sea conditions
over time.
109

Water Level for Smooth Sea State
The water level detection for smooth seas reached a mean RMSE of 14 cm for the whole
dataset and a maximum RMSE of 210 cm for a single observation. Similar to the predictions
of all water levels and the water levels during calm sea conditions, the water levels during
smooth seas were also under-predicted for low- and high-tide, while the water levels during
mid-tide were mostly over-predicted by the regression model. This is reflected in their
distribution by water level (Fig. 71), their scatter plot (Fig. 72), as well as by their plot over
time (Fig. 73). What is noticeable is that the over-prediction for the mid-tide water levels was
prevalent mostly for water levels between 365 and 460 cm. For the rest of the mid-tide water
levels, whether the values were over- or under-predicted varied strongly with the range of
water levels they fell into.
Fig. 71: Distribution of predicted (orange) vs measured (blue) water levels in 2015 during
smooth sea conditions.
110

Fig. 72: Scatter plot of predicted vs measured water levels in 2015 during smooth sea
conditions.
Fig. 73: Predicted (red) and measured (blue) water levels in 2015 during smooth sea
conditions over time.
111

Water Level for Slight Sea State
The water level detection for slight seas reached a mean RMSE of 12 cm for the whole dataset
and a maximum RMSE of 216 cm for a single observation. The distribution of the predicted
and observed water levels for slight seas (Fig. 74) shows similarities to the other distribution
before, especially to that of smooth seas (Fig. 71). Low- and high-tide water levels were
under-predicted by the regression model, while mid-tide levels ranging from 400 to 500 cm
were over-predicted. For the mid-tide water levels above 500 cm, it depended on their
specific range in centimetres whether they were over- or underpredicted. Both the scatter
plot of the predicted vs actual water levels (Fig. 75), as well as the plot of the predicted and
measured values over time (Fig. 76), reflect this behaviour of data predicted by the regression
model.
Fig. 74: Distribution of predicted (orange) vs measured (blue) water levels in 2015 during
slight sea conditions.
112

Fig. 75: Scatter plot of predicted vs measured water levels in 2015 during slight sea
conditions.
Fig. 76: Predicted (red) and measured (blue) water levels in 2015 during slight sea conditions
over time.
113

Water Level for Moderate Sea State
The water level detection for moderate seas reached a mean RMSE of 1 cm for the whole
dataset and a maximum RMSE of 172 cm for a single observation. The dataset for moderate
seas contained only a few observations where the water level was below 500 cm, while most
of the observations were available for the range from 550 to 700 cm. The distribution of the
water levels (Fig. 77) shows that the water levels between 600 and 700 cm were
over-predicted, while the other ranges were under-predicted. The exception is the range from
450 to 500 cm, where the prediction was seemingly perfect.
Fig. 77: Distribution of predicted (orange) vs measured (blue) water levels in 2015 during
moderate sea conditions.
However, looking at the scatter plot of the predicted vs measured values (Fig. 78), the water
levels in the 450 to 500 cm range were not perfectly predicted, but over-predicted instead.
Similar to the other sea state models, water levels at the lower and upper end of the total
range from the given observations are over- or underpredicted, respectively. The plot of
observations over time (Fig. 79) reflects this as well.
114

Fig. 78: Scatter plot of predicted vs measured water levels in 2015 during moderate sea
conditions.
Fig. 79: Predicted (red) and measured (blue) water levels in 2015 during moderate sea
conditions over time.
115

4.5 Predictions with 2016 Dataset
4.5.1 Wave Height Prediction
The wave height detection reached a mean RMSE of 17 cm for the whole dataset and a
maximum RMSE of 154 cm for a single observation. The distribution of wave heights in 2016
(Fig. 80) shows that the wave heights have been over-predicted by the regression model, and
somehow, the regression model even predicted negative wave heights. For the ranges 40 to
45 cm, 55 to 60 cm, and 70 to 75 cm the predictions were seemingly perfect. The wave
heights above 60 cm were all under-predicted by the model, and starting from a wave height
above 175 cm, the models did not predict these values at all.
Fig. 80: Distribution of the measured (blue) vs predicted (orange) wave heights in 2016.
The scatter plot (Fig. 81) reflects this behaviour; the predictions of the high wave heights all
fall below the regression line, showing that the true values were under-predicted.
Furthermore, the predictions are scattered around the perfect prediction diagonal;
comparing this result to that of 2015, the 2016 prediction performed worse. The plot of the
calculated and observed values (Fig. 82) also shows that the former are placed at lower wave
heights than the true latter.
116

Fig. 81: Scatter plot of predicted vs measured wave heights in 2016.
Fig. 82: Measured (blue) and predicted (red) wave heights in 2016 plotted over time.
117

4.5.2 Water Level Prediction
The water level detection reached a mean RMSE of 36 cm for the whole dataset and a
maximum RMSE of 209 cm for a single observation. The distribution of the 2016 water levels
(Fig. 83) shows under-prediction for water levels ranging below 395 cm, while the water
levels above 395 cm and below 505 cm were over-predicted. In the range from 505 to 610 cm,
over- and under-prediction vary, for the range from 610 to 665 cm the values were
under-predicted, and most of the water levels above 665 cm were over-predicted by the
regression model.
Fig. 83: Distribution of the measured (blue) vs predicted (orange) water levels in 2016.
The plot over time for the whole observation period (Fig. 84) reflects this prediction
behaviour as well since the lower water levels ( < 400 cm) were not predicted correctly.
Looking at a randomly selected period of two weeks (Fig. 85), the tide curve of the predicted
values follows the real tide curve well for high- and mid-tide water levels but does not follow
the valleys of the low tides. In comparison, the result from 2015 followed the course of the
tide curve much more closely than the results for 2016 data presented here.
118

Fig. 84: Measured (blue) and predicted (red) water levels in 2016 plotted over time.
Fig. 85: Measured (blue) and predicted (red) water levels for a two-week period in 2016.
119

Water Level for Calm Sea State
The water level detection for calm seas reached a mean RMSE of 1 cm for the whole dataset
and a maximum RMSE of 169 cm for a single observation. The range of water levels between
400 and 550 cm recorded during calm conditions was strongly over-predicted by the
regression model, while the water levels below 400 and above 550 cm were under-predicted
(Fig. 86). Water levels below 350 and above 600 cm were not predicted at all.
Fig. 86: Distribution of predicted (orange) vs measured (blue) water levels in 2016 during
calm sea conditions.
In the scatter plot of predicted vs measured water levels (Fig. 87), the predictions are strongly
scattered around the regression line, and the point cloud seems to pass almost horizontally
through the plot. Lower values were over-predicted while higher values were
under-predicted. Interestingly, while the histogram shows that the water levels below 400 cm
were under-predicted, the scatter plot shows that they were over-predicted. Comparing the
plot of observations and predictions over time (Fig. 88) to the histogram and the scatter plot,
the lower water levels were over-predicted, which reflects the results of the scatter plot but
not that of the histogram.
120

Fig. 87: Scatter plot of predicted vs measured water levels in 2016 during calm sea
conditions.
Fig. 88: Predicted (red) and measured (blue) water levels in 2016 during calm sea conditions
over time.
121

Water Level for Smooth Sea State
The water level detection for smooth seas reached a mean RMSE of 22 cm for the whole
dataset and a maximum RMSE of 209 cm for a single observation. For the prediction of the
water levels during smooth sea conditions, the histogram (Fig. 89) shows over-prediction for
mid-tide ranges and under-prediction for both low- and high-tide ranges. Similar to the
results for calm seas, the scatter plot (Fig. 90) does not reflect the distribution in the
histogram exactly; while the histogram displays under-prediction for the low-tide water
levels, the scatter plot shows over-prediction. In comparison to the scatter plot of water
levels during calm sea conditions, the scatter plot here is more uniformly distributed around
the diagonal; however, this point cloud is broader and shorter than the one for the prediction
of the 2015 water levels for smooth seas. The plot over time (Fig. 91) reflects the findings of
the scatter plot (Fig. 90).
Fig. 89: Distribution of predicted (orange) vs measured (blue) water levels in 2016 during
smooth sea conditions.
122

Fig. 90: Scatter plot of predicted vs measured water levels in 2016 during smooth sea
conditions.
Fig. 91: Predicted (red) and measured (blue) water levels in 2016 during smooth sea
conditions over time.
123

Water Level for Slight Sea State
The water level detection for slight seas reached a mean RMSE of 12 cm for the whole dataset
and a maximum RMSE of 198 cm for a single observation. Similar to the aforementioned
water level predictions, the histogram plotting the distribution of the predicted vs measured
water levels (Fig. 92) shows under-prediction for low and high tide, and the mid-tide range is
subject to over-prediction. Here, water levels below 360 and above 680 cm were seemingly
not predicted by the model at all.
Fig. 92: Distribution of predicted (orange) vs measured (blue) water levels in 2016 during
slight sea conditions.
On the other hand, the scatter plot for the performance of predicted vs measured values (Fig.
93) shows that the lower measured values were over-predicted by the model; the placement
of the predictions in the plot over time (Fig. 94) reflects this behaviour as well. Therefore,
there is again a difference between the results presented by the distribution of ranges and
the results presented by comparing the predicted and measured values of an individual
observation to one another.
124

Fig. 93: Scatter plot of predicted vs measured water levels in 2016 during slight sea
conditions.
Fig. 94: Predicted (red) and measured (blue) water levels in 2016 during slight sea conditions
over time.
125

Water Level for Moderate Sea State
The water level detection for moderate seas reached a mean RMSE of 1 cm for the whole
dataset and a maximum RMSE of 146 cm for a single observation. The distribution for
moderate sea state water levels (Fig. 95) shows under-prediction for values below 600 cm and
over-prediction for the values above. Again, the scatter plot (Fig. 96) does not reflect this
distribution; there, the water levels above 650 cm were under-predicted while those below
600 cm were over-predicted. The plot over time (Fig. 97) agrees with the results from the
scatter plots, as the measured values below a water level of 600 cm were all miscalculated by
the regression model as higher than they were. On the other hand, the few observations of
water levels above 700 cm were all under-predicted by the regression model.
Fig. 95: Distribution of predicted (orange) vs measured (blue) water levels in 2016 during
moderate sea conditions.
126

Fig. 96: Scatter plot of predicted vs measured water levels in 2016 during moderate sea
conditions.
Fig. 97: Predicted (red) and measured (blue) water levels in 2016 during moderate sea
conditions over time.
127

5. Discussion
5.1 Image Preprocessing
The preprocessing step for the images performed well overall but still had weaknesses.
Especially, the performance for the correct detection of twilight images was low, which
resulted in daylight images being misclassified. In contrast to the misclassification of night
images as twilight images and vice versa, the misclassification of the daylight images is
problematic. The regression routine (see Appendix II: Regression with Interpolation)
automatically fetches the groyne images taken during daylight hours from the Cropped folder
and extracts features from them. However, if the daylight images are falsely sorted into
another folder, they will not be considered by the regression routine, thus resulting in a
smaller available dataset, which can affect the predictions negatively, as less relevant
features could be extracted. For wave heights and water levels that are generally
underrepresented in the dataset, this is a massive problem, as they might end up not being
reflected in the feature data at all.
The previous approach from Reuter (2022) for sorting images by their lighting conditions
involved one ROI, the sky, from which the mean of the grey values was extracted and
compared to different thresholds for daylight, twilight, and night (see 2.2 Preliminary Work).
However, since moonlight or artificial light can result in a mean grey value of the sky being
higher than the threshold for daylight images, this approach tended to sort many images into
the wrong categories. Therefore, as described in 3.3.1 Preprocessing the Image Data (section
Sort, Crop, and Save Images), the detection routine was appended by two other ROIs –
ground and ocean (see 3.3.1, Fig. 30) – to increase the likelihood of an image being sorted
into the correct category.
Another previously tested method for the sorting algorithm was to use the times of sunrise
and sunset for Noderney and sort all images that were taken between sunrise and sunset into
the Daylight category. Unfortunately, this approach produced errors frequently, and the
image dataset had to be sorted into monthly, weekly, and even daily categories as the script
would otherwise fail to complete the routine without errors. Additionally, since the UTC
128

coefficient needs to be taken into account, for a site like Norderney, where a time change
between CET (UTC+1) and CEST (UTC+2) is applied, the dataset generally needs to be sorted
into categories “during CET” and “during CEST”. Such a pre-sorting step is feasible, however,
it does introduce additional load to the pre-processing step. Furthermore, some images
produced an error for their correct UTC coefficient (e.g. UTCoff = 1) and sorting only worked
when its value was changed to that of the “false” coefficient (e.g. UTCoff = 2). While this
approach did not work as intended for the image dataset from Norderney, it might work
better for sites without a time change. However, since the sorting routine using sunrise and
sunset times was time-intensive and error-prone, it was ultimately disregarded as a
practicable option for sorting the webcam images from Norderney.
Fig. 98: Daylight image falsely sorted into the twilight category, taken on 4th February 2015,
10:30 UTC+1.
Above is an example of a daylight image that has been falsely sorted into the twilight
category (Fig. 98). In the image, the mean grey values of the sky and sea ROIs are high enough
to surpass the thresholds necessary to be sorted into the daylight category. However, there
are shadows cast by the buildings onto the boardwalk, which darken the area, which in turn
produces a low mean grey value for the ground ROI. Since this value remains below the
threshold for daylight images, the image is misclassified as a twilight image. In the case of
129

Fig. 1, the mean of the ground ROI was roughly 24.7 while the minimum mean needed to
enter the daylight category is 30.
Another type of daylight image which was frequently sorted into the twilight category was
images taken during cloudy skies (Fig. 99). For these images, the mean grey value of the sky
oen fell below the threshold for daylight images, as the cloud cover (e.g. with rain clouds)
lowered the mean grey value of the sky. The reflection ofthe dark clouds or the cast of cloud
shadows on the sea surface also lowered the mean grey value of the ocean ROI, however, not
enough to cause the misattribution. In the case of Fig. 99, the mean value for the sky was
roughly 148.5, while the threshold for daylight images is 150.
Fig. 99: Daylight image with cloudy skies, falsely sorted into the twilight category, taken on
15th June 2015, 10:00 UTC+2.
A sensible way of dealing with the false sorting of daylight images taken during sunshine
could be the implementation of a second ground ROI which is not covered by the shadows.
However, finding such a specific ROI is difficult, as the shadows move during the day with the
position of the sun. Therefore, another method of improving the sorting results needs to be
applied.
130

Currently, the sorting algorithm uses an OR connection of the mean values extracted from
the three ROIs. However, it might be more sensible to check if at least two thresholds for a
category are reached (see code below, with thresholds for the twilight class), and then sort
the images based on that criterion. In the case of the two misclassified daylight images
presented here (Fig. 98 + 99), the algorithm would have detected two criteria for the daylight
classes to be true and sorted them correctly. Using an AND connection for all three
thresholds would also be a possibility, but this might cause too many images taken during
daylight hours to be sorted into the wrong categories.
(meanSky < 150 && meanGround < 30) || (meanSky < 150 && meanOcean < 65) ||
(meanGround < 30 && meanOcean < 65)
The misclassification of twilight images as night images and vice versa is generally not much
of a problem, as the routine only uses daylight images. However, improvement of the
detection as aforementioned would open the possibility of using a larger dataset for
extracting features. For example, techniques such as contrast or low-light image
enhancement [64, 65] could be applied to the night and twilight images to transform them
into images usable by the feature extraction algorithm. The usage of enhancement
techniques, perhaps even in combination with deep learning [66], would not only enable the
usage of non-daylight images but could also be used as another step in the part of the
routine that crops and saves the daylight images. Using such a technique, the sorting
algorithm would not need to be able to detect every instance correctly, but misclassified
images could be enhanced to produce sufficient contrast between the groyne and its
surroundings instead.
An example of a night or twilight image that was falsely sorted into the daylight category is
Fig. 100, which was considered a daylight image by the algorithm due to all detection ROIs
producing higher means than the thresholds needed for daylight images (meanSky ≈ 153.0 >
150, meanOcean ≈ 70.1 > 65, and meanGround ≈ 55.9 > 30). The ground has a relatively high
mean value likely due to the artificial lights that illuminate the boardwalk, and, judging from
the light at the bottom le of the image, from building lights as well. While there are dark
clouds close to the camera, there is still a lot of bright sky near the horizon, which will have
131

produced a mean high enough to pass the daylight threshold. Lastly, since the ocean reflects
the light from the sky on its surface, its mean is higher than the threshold as well. Whilethe
ROI for the ocean mean calculation was positioned outside of the surf zone, it should be
noted that a large area of seafoam caused by breaking waves could cause images to be
misclassified as well.
Fig. 100: Night image (le) wrongly classified as daylight image and cropped (le), taken
on 27th January 2015 at 18:00 UTC+1.
Using the misclassified image from Fig. 100 and applying low-light enhancement [65], the
brightness and contrast of an image can be increased, for example, to highlight the groyne.
While the result for the original image (Fig. 101) looks promising, the result for the cropped
image (Fig. 102) shows over-enhancement, mainly in pixel clusters at the top of the groyne,
resulting in a less detailed and more pixelated resolution.
Fig. 101: Original (le) and low-light enhanced (right) night image.
132

Fig. 102: Original (le) and low-light enhanced (right) cropped night image.
Therefore, while enhancement as post-processing of the cropping step is possible, the
correct sorting of the images depending on lighting conditions has the highest priority. Any
improvement of the preprocessing step should aim to increase the number of correctly
identified daylight images, as they have the naturally occurring lighting conditions and
contrast to be used for feature extraction. The more daylight images are available for this, the
better the results of the prediction should become.
133

5.2 Classification Problem
Deep learning models can work with unstructured data (e.g. images) and extract features
independently; generally, they work better with large datasets. As the results in 4.2 show, the
performance of the same model on four classeswas 16.51 percentage points better thanthat
for 32 classes (from 69.91% to 86.42%), which makes sense, as there are more images
available per individual class for four than for 32 total classes. With more images per class,
the pool of data available for feature extraction is larger, allowing the model to generate
more features for the identification of a class. Since the model can learn on more feature data
per class, the classification performance increases.
Switching from a GoogLeNet to a ResNet18 resulted in an improvement of the validation
accuracy of nearly three percentage points (from 57% to 59.9888%). The GoogLeNet is a
convolutional neural network (CNN) based on the Inception architecture [67], which means
that it allows the use of multiple types of filter sizes in a single image block, instead of being
restricted to a single filter size [68]. The ResNet18 uses residual blocks, which counteract the
degradation problem, where networks converge and their accuracy gets saturated (compare
3.4.2 Network Example: ResNet18). The GoogLeNet likely performed slightly worse than the
ResNet18 because the filter size of the pretrained Inception blocks did not match the filter
size that would have been ideal for the new input data; when a GoogLeNet is trained, it
chooses the filter sizes depending on the input it receives [69].
The highest accuracy for 32 classes was achieved using difference images, which was 28.51
percentage points higher than the accuracy for the normal RGB images (from 69.91% to
98.42%). Difference images, as described in 3.3.3 Preprocessing Image Data for Classification,
section Image Types, only contain information that is different from other images (Fig. 103).
Parts of the images that are the same in other images of the same class (combined in an
average image) will be blacked out while the parts that are unique to the image will be
brighter. This means, if the model is confronted with difference images from different classes,
it can learn features from the differences only, rather than learning features from every part
of the image, even those that do not or negatively contribute to the differentiation between
the different classes.
134

Fig. 103: Two difference images taken during different conditions.
On the other hand, edge images performed 13.93 percentage points worse than the RGB
images for the classification problem with four classes. Looking at the edge images in Fig.
104, which were taken during calm (le) and moderate (right) seas, the detected edges do
not reflect these differences well. Since the edges were detected from the RGB images, all
edges that the edge filter can find within this image will be detected, including those that are
the same in all images regardless of the water level or sea state. The major problem with
edge images is that the edge filters find the most edges in images with high contrast, even if
these edges do not match the borders a human reviewer would consider to be edges.
Fig. 104: Two edge images taken during different conditions.
In Fig. 103 - 105, the le images were all taken on the 2nd August 2015 at 6:30UTC+2 during
calm seas with a water level below 350 cm, while the right images were taken on the 30th
March 2015 at 15:50 UTC+2 during moderate seas with a water level above 700 cm.
135

Fig. 105: RGB images which the edge images in Fig. 104 were derived from.
The le difference image (Fig. 103) has barely any spots that stand out against the
background, which means that there is little variation in this image class. For images taken
during calm seas and low water levels, this makes sense, as there are barely any dynamic
elements present in the pictures. The right difference image (Fig. 103) on the other hand has
many bright spots, especially at the border between the groyne and its surroundings, and in
the surf zone. Given that the image class contains images taken during moderate seas and
high water levels, there is a high intra-class variation, as the waves introduce a highly
dynamic element to the picture. The edge images (Fig. 104) however do not reflect the
overall class properties as well. It would be expected that few edges are detected for calm
seas and many for rougher seas, however, as the contrast is the deciding factor for an edge
being detected, the waves in the right image were not detected as edges. For the
classification, this is a problem, as there is a high chance that the feature “few edges present”
is interpreted as “no or few waves present”, resulting in misclassification of the images, as
the edges are the only element to derive features from.
The normal RGB images (Fig. 105) have the largest number of elements for the models to
derive features from, however, as the relevant elements are not highlighted as they are in the
difference images, the models can derive redundant features or even those that introduce
error to the prediction. The result is that difference images perform best, RGB images second
best, and edge images worst, reflecting the ability of the image types to contain relevant
information very well.
136

5.3 Regression Problem
5.3.1 Regression with Deep Learning
Between the modified ResNet18 and the network with two inputs, the former performed
better (see 4.3.1 Regression with Deep Learning). The main reason for this is most likely that
the modified ResNet18 still had most of its pretrained layers, meaning that the network did
not need to be trained from scratch. The double-input network instead was created from
scratch, which means that it had to learn features first before being able to make good
predictions. Furthermore, the double-input network was trained only for 15 epochs with a
total of 3,810 iterations and a training time of almost 127 minutes. Given that deep learning
networks are usually trained for much longer (weeks or even months [70]), it might be
appropriate to increase the epochs for training. The RMSE of the double-input network had
already been reduced from ca. 17*10e6 cm to 1.9*10e4 cm by the final iteration, so it is
plausible that the result might improve even further with a longer training time.
While the ResNet18 did perform significantly better than the network with image and feature
inputs, the issue that remains is that its RMSE began to plateau aer 500 of almost 3,000
iterations. The problem is to find out what may have caused the RMSE to be saturated. One
candidate is the network itself, as perhaps its layers did not detect the optimal features for
the predictions. Another candidate might be the data itself, in the way that it does not
provide enough information for a better prediction. This, however, seems unlikely, as the
results of the machine learning approaches from the Regression Learner App did produce
lower RMSEs than the ResNet18. Therefore, it seems more likely that the limitation resulting
in the plateau lies with the structure or parameters of the ResNet18, rather than the data
itself. However, it is beyond the scope of this study to examine the entire ResNet18 and the
way it deals with the given data to find the ideal network structure and configuration for the
specific dataset.
137

5.3.2 Regression with Machine Learning
Looking at the results from regression with machine learning approaches and comparing
them to the results using deep learning, it is noticeable that the ML approach performed
better for wave height detection than the DL approach did. A possible explanation for the ML
model performing better than the DL network is the format of the input data they receive and
the way it is used for training. While the ML models are provided with feature data that was
previously selected and extracted by the user, the DL networks have to find and extract
relevant features by themselves. The automatic extraction of features reduces the workload
for the user, however, it is also a point where errors can be introduced early on. If the DL
network extracts features that are redundant or perhaps even those that negatively affect the
prediction, then these self-selected features can result in a worse performance than those
selected by the user.
Furthermore, the ML models that produced the lowest training RMSEs in the Regression
Learner App are all based on Gaussian Process Regression (GPR). This type of regression is
known for having a reduced uncertainty at and around the observations (compare 3.5.2
Model Example: Gaussian Process Regression (GPR)), which means that every set of features
with a known response improves the detection function, increasing the likelihood of new
data being predicted correctly due to likeliness to data the model had previously been
trained on.
Since the prediction of the water level during calm seas was the only GPR model that used
the Matern 5/2 kernel function of the Mátern class, rather than the exponential kernel that
the other models used, it can be inferred that its stochastic process is not as smooth as that
of the others. The rate of decay of the power spectrum S(s) gives important information
about the smoothness of the associated stochastic process, while the rate of decay of the
eigenvalues reveals important information about kernel smoothness. Since “rougher”
processes have more power at high frequencies, their eigenvalue spectrum decays more
slowly; this is reflected in the power spectrum of the Mátern class (Rasmussen & Williams,
2005c).
138

The features used for the regression task were inferred from properties of the grey-level
co-occurrence matrix (GLCM); here, the mean, range, and variance as well as the so-called
“graycoprops” contrast, correlation, energy, and homogeneity were used. For the 8-by-8
GLCMs, the first three features returned n-by-8 arrays, while the latter four returned n-by-1
vectors, resulting in an n-by-28 feature array. The mean returned the average value for each
column of the GLCM, the range returned the difference between the minimum and maximum
value of each column, and the variance returned how far each value in the column was
spread from their mean. The contrast measures the local variations in the GLCM, the
correlation measures the joint probability occurrence of the specified pixel pairs, the energy
returns information about the uniformity of the GLCM, and the homogeneity measures the
closeness of the distribution of elements in the GLCM to the GLCM diagonal [71].
Other feature options that were tested within the scope of this thesis were: (1) only the
graycoprops, (2) only mean, range, and variance, (3) using the mean of the mean values, the
range of the ranges, and the variance of the variances, and (4) using the mean, range, and
variance with the standard deviation, the RMS, the mode (most frequent values), the
minimum and the maximum. All of these performed significantly worse than the 28 feature
columns described above. While further features or feature combinations were not tested, it
would be sensible for further research to focus on which features need to be added as input
for the models, as the image features have a higher influence on the model performance than
its parameters. Furthermore, the extraction of features with an algorithm or perhaps even a
deep learning approach is faster to implement than optimization, as it can be very time
intensive to calculate the ideal hyperparameters. Other texture features that could improve
the predictions when included are the global and local entropy of a grayscale image, and the
local range and standard deviation of an image [72].
In the scope of this thesis, the models were trained on the 2015 dataset, and predictions were
done for both the 2015 and 2016 datasets, separately. Comparing the results from 4.4
Predictions with 2015 Dataset and 4.5 Predictions with 2016 Dataset to one another, it is
noticeable that the 2016 dataset was not as well predicted by the models as the 2015
datasets. This was most visible in the scatter plot, where the scatter point clouds were rather
widely spread around the regression line for the 2016 predictions, while those for the 2015
139

predictions deviated less from the regression line. This result is not surprising,as the models
were trained on the 2015 dataset, which means that the features extracted from the 2016
images did not match the features the models had been trained on as closely as those
extracted from the 2015 images. While it might seem surprising that the water levels and
wave heights from the 2015 dataset were not predicted with 100% accuracy and an RMSE of
zero, it does make sense. During training, it is not the goal to find a function for the GPR that
fits the given observations with an accuracy of 100%, as this would mean that any data that
does not match the training data exactly would be miscalculated, which presents an
overfitting problem. If the models were able to predict the water levels and wave heights of
the 2015 dataset with an accuracy of 100%, it would be an indicator that the trained model
has specialised itself to only produce correct results with the exact features it had been
trained on. Therefore, the results from the 2015 and 2016 datasets show that the model is
general enough to work with similar (e.g. same site and angle of view) but not the same (e.g.
weather and wave conditions) new data.
The preliminary work by Reuter (2022) achieved an RMSE of 38.7 cm for the water level
prediction for the time frame from the 1st of January 2015 to the 14th of October 2016. Here,
an RMSE of 26 cm for the 2015 dataset and an RMSE of 36 cm for the 2016 dataset were
achieved. If the mean of these two RMSEs is computed without taking normalisation over the
number of observations (the 2015 dataset had more observations than that of 2016) into
account, then an RMSE of 31 cm for the water level detection was achieved. However, if the
models had been trained on the full dataset, rather than only the 2015 dataset, the RMSE
might have been even lower. It should be noted however that the models for water level
predictions were sorted by sea state. For 2015 data, the prediction of water levels for calm
seas performed best, with an RMSE of 0 cm, while the prediction for smooth seas with an
RMSE of 14 cm performed worst. For 2016 data, the lowest RMSE was 1 cm for calm seas and
22 cm for smooth seas.
For the wave height prediction, instead of an RMSE the coefficient of determination R2= 0.238
was given. Since the model for the wave height prediction was retrained a few times, the last
available value for R2from the Regression Learner App was R2= 0.65 with an RMSE of 18 cm
(the current RMSE for wave height prediction is 11 cm) for the 2015 dataset. R-squared
140

indicates how much variation of a dependent variable is explained by the independent
variable(s) in a regression model, and ranges from 0 (no variation is explained) to 1 (all
variation is explained) [73]. This means that for Reuterʼs approach, 23.8% of the predicted
wave heights were within the variance of the measured wave heights, while for the approach
presented in this thesis at least 65% of the predicted values are found in the variance of the
measured values. Comparing the scatter plots of the wave height predictions (Fig. 106 + 107),
it is noticeable that the predictions with Reuterʼs approach are more spread around the
regression line than the predictions achieved with the approach presented in this thesis.
Fig. 106: Scatter plot for water levels from 2015 and 2016 (Reuter, 2022).
Fig. 107: Scatter plots for water levels from 2015 (le) and 2016 (right).
141

6. Conclusion
This master thesis built routines which make image data usable for regression problems. Two
main routines were created: one for pre-processing the images, and the other for performing
regression using a machine-learning approach.
The pre-processing routine (Appendix I) sorts the raw images into different categories
depending on the lighting conditions present in the images and crops the images taken
during daylight hours to the ROI, which is the right groyne in the images. In addition, a
linking table that connects the images with the measured data is created. The linking table
contains the paths to the images and their timestamps, the water levels measured at the
same time, measured and interpolated wave heights, lighting conditions, and whether or not
the beach is visible in the images. Furthermore, gaps are removed from the measured data.
Pre-processing is needed as it reduces the image to the ROI only, rather than including more
information than necessary to predict the water levels and wave heights correctly.
The regression routine (Appendix II) utilises the cropped daylight images from the
pre-processing step to extract features for the regression task. The RGB images are converted
to grayscale, then their grey-level co-occurrence matrix (GLCM) is computed, and features are
derived from this matrix. For training purposes, it is possible to combine the features with a
response variable (wave height or water level) to retrain the model on new data. This routine
first predicts the wave heights from the image feature, then it will predict the water levels
depending on the sea state category the images belong to based on the wave height
prediction. Since waves can be interpreted as noise on the water levels, the splitting of the
regression models was done to improve the results of the water level prediction.
As described in the previous chapters, both the pre-processing and the regression routines
used in this thesis performed better than the routines from the preliminary work. While there
were outliers in the sorting step of the pre-processing routine, most of the images were
correctly sorted by their lighting conditions. However, further improvement of this step could
produce better results for feature detection, and in turn for the regression task. The wave
detection from the approach of the preliminary work resulted in a determination coefficient
142

R2of 0.238, which means that 23.8% of the predicted wave heights were within the variance
of the measured wave heights. The last available value for R-squared of the wave height
prediction was 0.65 at an RMSE of 18 cm, which means that 65% of the predicted values were
found in the variance of the measured values; however, the currently best result for the
calculation of the wave height was an RMSE of 11 cm, meaning that the R2-value might be
even higher for the dataset.
The preliminary work achieved an RMSE of 39 cm for the calculation of the water levels
between the 1st of January 2015 and the 14th of October 2016. The approach from this thesis
achieved an RMSE of 26 cm for the 2015 dataset and an RMSE of 36 cm for the 2016 dataset.
However, these values do not reflect the performance of the models accurately, as there is
not only one model that predicts all water levels butthere are four models which predict the
water level sorted by the predicted wave height. For the 2015 dataset, the models returned
the following RMSEs: calm seas with 0 cm, smooth seas with 14 cm, slight seas with 12 cm,
and moderate seas with 1cm. The following RMSEs were achieved on the 2016 dataset: 1 cm
for calm seas, 22 cm for smooth seas, 12 cm for slight seas, and 1 cm for moderate seas.
Therefore it can be concluded that the approach for the water level prediction presented in
this thesis produced more accurate results than the approach from the preliminary work.
Gaussian process regression (GPR) is used in machine learning (ML), which is part of the
broader category of artificial intelligence (AI). Therefore, GPR can be considered an AI-based
approach in the broader sense. Features were extracted from the images and combined with
measured parameters to create training data for the models. Using feature extraction, the
models are capable of predicting water levels and wave heights from images. Since the
model trained on the 2015 dataset did not produce drastically higher errors when confronted
with the 2016 dataset, the generalisation to apply the model to images from the same
camera view is given. While outliers are producing high individual RMSEs, the global RMSEs
of the models are low enough that they are usable for realistic applications. A pre-processing
routine for the images was created, and the option to retrain a model is given in the
regression script.
143

Compared to other state-of-the-art routines, the here presented approach includes the
pre-processing step, which is necessary to improve the data selection the feature extraction
draws on. Furthermore, many approaches in the coastal engineering and maritime science
field do not rely on webcam images for the parameters they aim to analyse but use satellite
or Lidar images instead. While these images contain information the webcam images cannot
provide, they are expensive to procure and their temporal resolution is lower than that of the
webcam images. In addition, the here presented approach detects both water levels and
wave heights; many of the reviewed papers would focus on only one ofthe two parameters.
Properties of the GLCM, in combination with its mean, range, and variance, were used as
image features for the regression task.
The findings of this thesis are useful for applications where the goal is to predict or detect
parameters from images, and measurement data is present for the parameters as well.
Furthermore, while the models of the final regression routine are based on ML and GPR, the
detailed description of the building and training of the deep learning (DL) approaches makes
it possible to recreate these models for regression or classification tasks.
Future research that may build on this thesis should focus on how the presented results
could be improved further, as already discussed in the previous chapter. The pre-processing
routine could be improved to correctly detect more daylight images, as they are the base of
the feature extraction. It should be considered which additional features might improve the
prediction performance, to lower the RMSEs further. While the DL approaches were not used
for the final routines, it should be considered if there are parts in the routine that could make
use of DL, e.g. enhancement of twilight images to increase the available data volume for
feature extraction. Furthermore, it should be tested if the RMSE of the dual-input network
could improve over more iterations, or how the RMSE plateau of the ResNet18 could be
prevented.
It should be noted that it was beyondthe scope of this thesis to create a model that can deal
with images from other sites or with a different angle of view without the need for retraining
the model on the new data. Finally, morphological changes were not the subject of this thesis
as webcam images can only detect the presence of changes but are unable to quantify them.
144

References
Scientific Publications
Behre, K.-E., & van Lengen, H. (1995). Ostfriesland: Geschichte und Gestalt einer
Kulturlandscha (p. 173). Aurich: Ostfries. Landscha. https://d-nb.info/943717205.
Chaudhary, P., DʼAronco, S., Leitão, J. P., Schindler, K., & Wegner, J. D. (2020). Water level
prediction from social media images with a multi-task ranking approach. ISPRS
Journal of Photogrammetry and Remote Sensing,167, 252–262.
https://doi.org/10.1016/j.isprsjprs.2020.07.003.
Chaudhary, P., DʼAronco, S., Moy de Vitry, M., Leitão, J. P., & Wegner, J. D. (2019). Flood-water
level estimation from social media images. ISPRS Annals of the Photogrammetry,
Remote Sensing and Spatial Information Sciences,IV-2/W5, 5–12.
https://doi.org/10.5194/isprs-annals-iv-2-w5-5-2019.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Li, F.-F. (2009, June). ImageNet: A large-scale
hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern
Recognition.http://dx.doi.org/10.1109/cvpr.2009.5206848.
Ebden, M. (2008, August). Gaussian processes: A quick introduction. arXiv.Org.
https://arxiv.org/abs/1505.02965.
Goldstein, E. B., Coco, G., & Plant, N. G. (2019). A review of machine learning applications to
coastal sediment transport and morphodynamics. Earth-Science Reviews,194,
97–108. https://doi.org/10.1016/j.earscirev.2019.04.022.
Guo, S., Zhang, Y., & Liu, Y. (2020). A water-level measurement method using sparse
representation. Automatic Control and Computer Sciences,54(4), 302–312.
https://doi.org/10.3103/s0146411620040069.
Gutierrez, B. T., Plant, N. G., Thieler, E. R., & Turecek, A. (2015). Using a Bayesian network to
predict barrier island geomorphologic characteristics. Journal of Geophysical
Research: Earth Surface,120(12), 2452–2475. https://doi.org/10.1002/2015jf003671.
145

Hies, T., Parasuraman, S. B., Wang, Y., Duester, R., Eikaas, H. S., & Meng, T. K. (2012). Enhanced
water-level detection by image processing. 10th International Conference on
Hydroinformatics.
https://www.researchgate.net/publication/262337135_Enhanced_water-level_detect
ion_by_image_processing.
Hipwell, B., & Alexander, L. (2022, January 18). An introduction to AI. Reddie &amp; Grose LLP.
https://www.lexology.com/library/detail.aspx?g=4fa5406f-4533-4669-97ab-03b81e3f
d71a.
Hoonhout, B. M., Radermacher, M., Baart, F., & van der Maaten, L. J. P. (2015). An automated
method for semantic classification of regions in coastal images. Coastal Engineering,
105, 1–12. https://doi.org/10.1016/j.coastaleng.2015.07.010.
Ioffe, S., & Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by
Reducing Internal Covariate Shi. Journal of Machine Learning Research,37.
https://proceedings.mlr.press/v37/ioffe15.html.
Jäger, W. S., Christie, E. K., Hanea, A. M., den Heijer, C., & Spencer, T. (2018). A Bayesian
network approach for coastal risk analysis and decision making. Coastal Engineering,
134, 48–61. https://doi.org/10.1016/j.coastaleng.2017.05.004.
Jörges, C., Berkenbrink, C., & Stumpe, B. (2021). Prediction and reconstruction of ocean wave
heights based on bathymetric data using LSTM neural networks. Ocean Engineering,
232, 109046. https://doi.org/10.1016/j.oceaneng.2021.109046.
Kunz, H. (1991, May 20). Artificial beach nourishment on norderney, A case study. Coastal
Engineering 1990.http://dx.doi.org/10.1061/9780872627765.249.
Mittelstaedt, E. (2003). Nord- und Ostsee: Gezeiten, Strömungen, Wasserschichtung. In
Nationalatlas Bundesrepublik Deutschland / Band 2 – Natur und Umwelt I: Relief,
Boden und Wasser (pp. 118–119). Elsevier, Spektrum, Akad. Verl.
http://archiv.nationalatlas.de/wp-content/art_pdf/Band2_118-119_archiv.pdf.
Niemeyer, H. D. (1995, August 11). Long-Term morphodynamical development of the East
Frisian islands and coast. Coastal Engineering 1994.
http://dx.doi.org/10.1061/9780784400890.176.
Niemeyer, H. D., Kaiser, R., & Knaack, H. (1997, August 5). Effectiveness of a combined beach
and shoreface nourishment on the Island of Norderney/East Frisia, Germany. Coastal
Engineering 1996.http://dx.doi.org/10.1061/9780784402429.360.
146

Plomaritis, T. A., Costas, S., & Ferreira, Ó. (2018). Use of a Bayesian Network for coastal
hazards, impact and disaster risk reduction assessment at a coastal barrier (Ria
Formosa, Portugal). Coastal Engineering,134, 134–147.
https://doi.org/10.1016/j.coastaleng.2017.07.003.
Poelhekke, L., Jäger, W. S., van Dongeren, A., Plomaritis, T. A., McCall, R., & Ferreira, Ó. (2016).
Predicting coastal hazards for sandy coasts with a Bayesian Network. Coastal
Engineering,118, 21–34. https://doi.org/10.1016/j.coastaleng.2016.08.011.
Rasmussen, C. E., & Williams, C. K. I. (2005a). Chapter 1: Introduction. In Gaussian Processes
for Machine Learning. MIT Press. http://gaussianprocess.org/gpml/chapters/RW1.pdf.
Rasmussen, C. E., & Williams, C. K. I. (2005b). Chapter 2: Regression. In Gaussian Processes for
Machine Learning. MIT Press. http://gaussianprocess.org/gpml/chapters/RW2.pdf.
Rasmussen, C. E., & Williams, C. K. I. (2005c). Chapter 4: Covariance Functions. In Gaussian
Processes for Machine Learning. MIT Press.
http://gaussianprocess.org/gpml/chapters/RW4.pdf.
Rasmussen, C. E., & Williams, C. K. I. (2005d). Chapter 5: Model Selection and Adaptation of
Hyperparameters. In Gaussian Processes for Machine Learning. MIT Press.
http://gaussianprocess.org/gpml/chapters/RW5.pdf.
Rasmussen, C. E., & Williams, C. K. I. (2005e). Chapter 7: Theoretical Perspectives. In Gaussian
Processes for Machine Learning. MIT Press.
http://gaussianprocess.org/gpml/chapters/RW7.pdf.
Reuter, B. (2022). Analysis of Webcam Footage from Norderney Beach [Bachelorʼs Thesis].
Ludwig-Franzius-Institut Hannover.
Schlütz, F., Enters, D., & Bittmann, F. (2021). From dust till drowned: The Holocene landscape
development at Norderney, East Frisian Islands. Netherlands Journal of Geosciences,
100(7). https://doi.org/10.1017/njg.2021.4.
Thorenz, F., & Blum, H. (2011). Implementing coastal defence strategies for sandy coasts -
reinforcement of the Norderney dune revetment. 5th SCACR 2011 International Short
Conference on Applied Coastal Research, 238–243.
https://www.researchgate.net/publication/358834544_Implementing_coastal_defen
ce_strategies_for_sandy_coasts_-_reinforcement_of_the_Norderney_dune_revetme
nt.
147

Vandaele, R., Dance, S. L., & Ojha, V. (2021). Automated water segmentation and river level
detection on camera images using transfer learning. In Lecture Notes in Computer
Science (pp. 232–245). Springer International Publishing.
http://dx.doi.org/10.1007/978-3-030-71278-5_17.
Wang, J. H., Li, G., Xiong, Y. Z., & Liu, K. K. (2013). Method of Detecting Sea State from Image
Taken by Autonomous Surface Vehicle. Applied Mechanics and Materials,316–317,
475–478. https://doi.org/10.4028/www.scientific.net/amm.316-317.475.
148

Weblinks
[1] World Meteorological Organization. (2021, January 14). 2020 was one of three warmest
years on record. World Meteorological Organization. Retrieved January 29, 2023, from
https://public.wmo.int/en/media/press-release/2020-was-one-of-three-warmest-yea
rs-record.
[2] United Nations. (n.d.). What is climate change? United Nations. Retrieved January 29,
2023, from https://www.un.org/en/climatechange/what-is-climate-change/.
[3] Lindsey, R. (2022). Climate change: Global sea level. In NOAA Climate.gov. Retrieved
January 29, 2023, from
https://www.climate.gov/news-features/understanding-climate/climate-change-glob
al-sea-level.
[4] tagesschau. (2022, February 20). Neues Sturmtief im Anmarsch: Auf “Ylenia” und “Zeynep”
folgt “Antonia.” Tagesschau.De. Retrieved January 29, 2023, from
https://www.tagesschau.de/inland/gesellscha/sturm-antonia-101.html.
[5] Hannoversche Allgemeine Zeitung. (2022, February 19). Nordsee-Inseln: Sturm “Zeynep”
spült Strände fast komplett weg. Hannoversche Allgemeine Zeitung.
https://www.haz.de/der-norden/nordsee-inseln-sturm-zeynep-spuelt-straende-fast-k
omplett-weg-7NYUVODMNRC4NPXRDA6JPAQYKI.html.
[6] Niedersächsischer Landesbetrieb für Wasserwirtscha, Küsten- und Naturschutz
(NLWKN). (2011). Küstenschutz für die Insel Baltrum: Umgestaltung des Deckwerkes
am Nordstrand. Retrieved January 17, 2023, from
http://www.baltrum-online.de/pdf/Broschuere_Kuestenschutz_Baltrum.pdf.
[7] NLWKN. (July 12, 2019). Strandaufspülung auf Norderney: Maßnahmenstart in der
nächsten Woche. Nds. Landesbetrieb Für Wasserwirtscha, Küsten- Und Naturschutz.
Retrieved January 17, 2023, from
https://www.nlwkn.niedersachsen.de/startseite/aktuelles/presse_und_offentlichkeit
sarbeit/pressemitteilungen/strandaufspulung-auf-norderney-massnahmenstart-in-d
er-nachsten-woche-178779.html.
149

[8] NLWKN. (2019, August 21; last updated on 23-Aug-2019). Strandaufspülung auf
Norderney abgeschlossen. Nds. Landesbetrieb Für Wasserwirtscha, Küsten- Und
Naturschutz. Retrieved January 17, 2023, from
https://www.nlwkn.niedersachsen.de/startseite/aktuelles/presse_und_offentlichkeit
sarbeit/pressemitteilungen/strandaufspulung-auf-norderney-abgeschlossen-179887.
html.
[9] Aunkofer, B. (2018, May 14). Machine Learning vs Deep Learning - Wo liegt der Unterschied?
Data Science Blog. Retrieved January 21, 2023, from
https://data-science-blog.com/blog/2018/05/14/machine-learning-vs-deep-learning-
wo-liegt-der-unterschied/.
[10] Sharma, R. (2021, January 29). The ultimate data science cheat sheet every data scientists
should have. UpGrad. Retrieved January 21, 2023, from
https://www.upgrad.com/blog/data-science-cheat-sheet/.
[11] Schroer, A. (2022, September 19). What is artificial intelligence (AI)? How does AI work?
Built In. Retrieved January 21, 2023, from https://builtin.com/artificial-intelligence.
[12] Wuttke, L. (2022, April 26). Machine Learning: Algorithmen, Methoden und Beispiele.
Datasolut GmbH. Retrieved January 21, 2023, from
https://datasolut.com/was-ist-machine-learning/.
[13] Wuttke, L. (2021, October 30). Unsupervised Learning: Definition, Arten & Beispiele -
Datasolut Wiki. Datasolut GmbH. Retrieved January 21, 2023, from
https://datasolut.com/wiki/unsupervised-learning/.
[14] Wuttke, L. (2022, April 27). Reinforcement Learning: KI reagiert auf Belohnungen.
Datasolut GmbH. Retrieved January 21, 2023, from
https://datasolut.com/reinforcement-learning/.
[15] Wuttke, L. (2021, October 30). Supervised Learning: Definition, Arten & Beispiele -
Datasolut Wiki. Datasolut GmbH. Retrieved January 21, 2023, from
https://datasolut.com/wiki/supervised-learning/.
[16] IBM. (n.d.). Was sind neuronale Netze? IBM. Retrieved January 21, 2023, from
https://www.ibm.com/de-de/topics/neural-networks.
150

[17] Libretexts. (2018, June 11). 1.3: Sigmoid neurons. Libretexts. Retrieved January 21, 2023,
from
https://eng.libretexts.org/Bookshelves/Computer_Science/Applied_Programming/B
ook:_Neural_Networks_and_Deep_Learning_(Nielsen)/01:_Using_neural_nets_to_re
cognize_handwritten_digits/1.03:_Sigmoid_neurons.
[18] Wuttke, L. (2022, April 27). Machine Learning vs. Deep Learning: Wo ist der Unterschied?
Datasolut GmbH. Retrieved January 21, 2023, from
https://datasolut.com/machine-learning-vs-deep-learning/.
[19] OpenEarth, & Jäger, W. S. (2017). GitHub - Openearth/coastal-dss: Decision support
framework for coastal risk management: A Bayesian Network approach. GitHub.
Retrieved January 21, 2023, from https://github.com/openearth/coastal-dss.
[20] Chaves, M. (2022, January 31). GLCMs — a great tool for your ML arsenal. Towards Data
Science. Retrieved January 22, 2023, from
https://towardsdatascience.com/glcms-a-great-tool-for-your-ml-arsenal-7a59f1e45b
65.
[21] The MathWorks, Inc. (n.d.). Texture analysis using the gray-level co-occurrence matrix
(GLCM) - MATLAB & simulink - mathworks deutschland. MathWorks® Help Center.
Retrieved January 22, 2023, from
https://de.mathworks.com/help/images/texture-analysis-using-the-gray-level-co-occ
urrence-matrix-glcm.html.
[22] Keuschnig, G., & Radlherr, F. (2015, August 15). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 13,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/08/15/1450.
[23] Datawell BV. (2022, September 4). Directional Waverider MkIII. Datawell. Retrieved
January 25, 2023, from https://datawell.nl/products/directional-waverider-mkiii/.
[24] Hartmann, E. (n.d.). Norderney Pegelmesser. Deutsche Leuchtfeuer. Retrieved January
25, 2023, from
https://www.deutsche-leuchtfeuer.de/nordsee/norderney-pegelmesser.html.
[25] Chaves, M. (2022, January 31). GLCMs — a great tool for your ML arsenal. Towards Data
Science. Retrieved January 26, 2023, from
https://towardsdatascience.com/glcms-a-great-tool-for-your-ml-arsenal-7a59f1e45b
65.
151

[26] The MathWorks, Inc. (n.d.). Properties of gray-level co-occurrence matrix - MATLAB
graycoprops - MathWorks Deutschland. MathWorks® Help Center. Retrieved January
26, 2023, from https://de.mathworks.com/help/images/ref/graycoprops.html.
[27] Jackson Parton Solicitors. (n.d.). The Douglas Sea State Scale . Jackson Parton. Retrieved
January 26, 2023, from https://jacksonparton.com/the-douglas-sea-state-scale.
[28] C3.ai, Inc. (2020, November 10). Root mean square error (RMSE). C3 AI. Retrieved January
26, 2023, from https://c3.ai/glossary/data-science/root-mean-square-error-rmse/.
[29] Hargrave, M. (2022, December 19). Standard deviation formula and uses vs. variance.
Investopedia. Retrieved January 26, 2023, from
https://www.investopedia.com/terms/s/standarddeviation.asp.
[30] The MathWorks, Inc. (n.d.). Find edges in 2-D grayscale image - MATLAB edge - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 30, 2023, from
https://de.mathworks.com/help/images/ref/edge.html#d124e98982.
[31] Mun, G. C. (2016, January 10). Average image of a folder of images. MATLAB Central.
Retrieved January 26, 2023 from
https://de.mathworks.com/matlabcentral/answers/263278-average-image-of-a-folde
r-of-images.
[32] Donges, N. (2019, June 16). What is transfer learning? Exploring the popular deep learning
approach. Built In. Retrieved January 20, 2023, from
https://builtin.com/data-science/transfer-learning.
[33] Nandepu, R. (2019, November 16). Understanding and implementation of Residual
Networks (ResNets). Analytics Vidhya (Medium). Retrieved January 20, 2023, from
https://medium.com/analytics-vidhya/understanding-and-implementation-of-residu
al-networks-resnets-b80f9a507b9c.
[34] Gupta, D. (2020, January 29). Activation functions. Analytics Vidhya. Retrieved January
20, 2023, from
https://www.analyticsvidhya.com/blog/2020/01/fundamentals-deep-learning-activa
tion-functions-when-to-use-them/.
[35] Shah, D. (2023, January 3). The essential guide to data augmentation in deep learning.
V7labs. Retrieved January 20, 2023, from
https://www.v7labs.com/blog/data-augmentation-guide.
152

[36] StatSo Europe. (n.d.). Overfitting (Überanpassung). StatSo. Retrieved January 20,
2023, from https://www.statso.de/glossary/O/Overfitting.htm.
[37] The MathWorks, Inc. (n.d.). Deep Network Designer App - MATLAB & Simulink - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 20, 2023, from
https://de.mathworks.com/help/deeplearning/deep-network-designer-app.html.
[38] The MathWorks, Inc. (n.d.). ResNet-18 convolutional neural network - MATLAB resnet18 -
MathWorks Deutschland. MathWorks® Help Center. Retrieved January 20, 2023, from
https://de.mathworks.com/help/deeplearning/ref/resnet18.html
[39] DataScientest. (2021, December 22). Convolutional Neural Network (CNN). Weiterbildung
Data Science | datascientest.com. Retrieved January 20, 2023, from
https://datascientest.com/de/convolutional-neural-network-2
[40] Nandepu, R. (2019, November 16). Understanding and implementation of Residual
Networks (ResNets). Analytics Vidhya (Medium). Retrieved January 20, 2023, from
https://medium.com/analytics-vidhya/understanding-and-implementation-of-residu
al-networks-resnets-b80f9a507b9c
[41] The MathWorks, Inc. (n.d.). List of Deep Learning Layers - MATLAB & Simulink - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 20, 2023, from
https://de.mathworks.com/help/deeplearning/ug/list-of-deep-learning-layers.html
[42] Metabase. (n.d.). Data normalization. Metabase | Business Intelligence, Dashboards, and
Data Visualization. Retrieved January 20, 2023, from
https://www.metabase.com/learn/databases/normalization
[43] The MathWorks, Inc. (n.d.). 2-D Convolutional Layer - MATLAB - MathWorks Deutschland.
MathWorks® Help Center. Retrieved January 20, 2023, from
https://de.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.convolution2dlayer
.html
[44] The MathWorks, Inc. (n.d.). Batch Normalization Layer - MATLAB - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 20, 2023, from
https://de.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.batchnormalizatio
nlayer.html
[45] The MathWorks, Inc. (n.d.). Rectified Linear Unit (ReLU) layer - MATLAB - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 20, 2023, from
https://de.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.relulayer.html
153

[46] The MathWorks, Inc. (n.d.). Max pooling layer - MATLAB - MathWorks Deutschland.
MathWorks® Help Center. Retrieved January 20, 2023, from
https://de.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.maxpooling2dlayer
.html
[47] Brownlee, J. (2019, April 21). A gentle introduction to pooling layers for convolutional
neural networks. MachineLearningMastery.Com. Retrieved January 20, 2023, from
https://machinelearningmastery.com/pooling-layers-for-convolutional-neural-netwo
rks/.
[48] Cook, A. (2017, April 9). Global average pooling layers for object localization. Retrieved
January 20, 2023, from
https://alexisbcook.github.io/2017/global-average-pooling-layers-for-object-localizat
ion/.
[49] The MathWorks, Inc. (n.d.). Fully connected layer - MATLAB - MathWorks Deutschland.
MathWorks® Help Center. Retrieved January 21, 2023, from
https://de.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlaye
r.html.
[50] The MathWorks, Inc. (n.d.). Somax Layer - MATLAB - MathWorks Deutschland.
MathWorks® Help Center. Retrieved January 21, 2023, from
https://de.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.somaxlayer.html
[51] The MathWorks, Inc. (n.d.). Transfer Learning Using Pretrained Network - MATLAB &
Simulink - MathWorks Deutschland. MathWorks® Help Center. Retrieved January 27,
2023, from
https://de.mathworks.com/help/deeplearning/ug/transfer-learning-using-pretrained
-network.html.
[52] The MathWorks, Inc. (n.d.). GoogLeNet Convolutional Neural Network - MATLAB googlenet
- MathWorks Deutschland. MathWorks® Help Center. Retrieved January 27, 2023, from
https://de.mathworks.com/help/deeplearning/ref/googlenet.html.
[53] The MathWorks, Inc. (n.d.). Pretrained Deep Neural Networks - MATLAB & Simulink -
MathWorks Deutschland. MathWorks® Help Center. Retrieved January 27, 2023, from
https://de.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-n
etworks.html.
154

[54] The MathWorks, Inc. (n.d.). Train Convolutional Neural Network for Regression - MATLAB &
Simulink - MathWorks Deutschland. MathWorks® Help Center. Retrieved January 27,
2023, from
https://de.mathworks.com/help/deeplearning/ug/train-a-convolutional-neural-netw
ork-for-regression.html.
[55] The MathWorks, Inc. (n.d.). List of Deep Learning Layers - MATLAB & Simulink - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 27, 2023, from
https://de.mathworks.com/help/deeplearning/ug/list-of-deep-learning-layers.html.
[56] The MathWorks, Inc. (n.d.). Train Network on Image and Feature Data - MATLAB & Simulink
- MathWorks Deutschland. MathWorks® Help Center. Retrieved January 27, 2023, from
https://de.mathworks.com/help/deeplearning/ug/train-network-on-image-and-featu
re-data.html.
[57] Twin, A. (2022, November 15). Understanding overfitting and how to prevent it.
Investopedia. Retrieved January 27, 2023, from
https://www.investopedia.com/terms/o/overfitting.asp.
[58] Kumar, A. (2018, February 12). Machine learning: Validation techniques. DZone. Retrieved
January 27, 2023, from
https://dzone.com/articles/machine-learning-validation-techniques.
[59] Nagpal, A. (2017, October 18). Decision tree ensembles- bagging and boosting. Towards
Data Science. Retrieved January 27, 2023, from
https://towardsdatascience.com/decision-tree-ensembles-bagging-and-boosting-26
6a8ba60fd9.
[60] Jaadi, Z. (2021, April 1). A step-by-step explanation of principal component analysis (PCA).
Built In. Retrieved January 27, 2023, from
https://builtin.com/data-science/step-step-explanation-principal-component-analysi
s.
[61] Mutual Information. (2021). Gaussian Processes [Video]. On YouTube. Retrieved January
10, 2023, from https://www.youtube.com/watch?v=UBDgSHPxVME.
[62] Deisenroth, M. (2017). ML Tutorial: Gaussian processes (Richard Turner) [Video]. On
YouTube. Retrieved January 10, 2023, from
https://www.youtube.com/watch?v=92-98SYOdlY.
155

[63] The MathWorks, Inc. (n.d.). Kernel (covariance) function options - MATLAB & simulink -
MathWorks Deutschland. MathWorks® Help Center. Retrieved January 28, 2023, from
https://de.mathworks.com/help/stats/kernel-covariance-function-options.html.
[64] The MathWorks, Inc. (n.d.). Contrast enhancement techniques - MATLAB & simulink
example - mathworks deutschland. MathWorks® Help Center. Retrieved January 28,
2023, from
https://de.mathworks.com/help/images/contrast-enhancement-techniques.html.
[65] The MathWorks, Inc. (n.d.). Low-Light image enhancement - MATLAB & simulink example -
mathworks deutschland. MathWorks® Help Center. Retrieved January 28, 2023, from
https://de.mathworks.com/help/images/low-light-image-enhancement.html.
[66] The MathWorks, Inc. (n.d.). Image processing using deep learning video. MathWorks®
Videos Und Webinare. Retrieved January 28, 2023, from
https://de.mathworks.com/videos/image-processing-using-deep-learning-15825419
77744.html.
[67] Papers with Code. (n.d.). Papers with code - GoogLeNet explained. Papers With Code.
Retrieved January 28, 2023, from https://paperswithcode.com/method/googlenet.
[68] Papers with Code. (n.d.). Papers with code - Inception module explained. Papers With
Code. Retrieved January 28, 2023, from
https://paperswithcode.com/method/inception-module.
[69] Anwar, A. (2022, January 22). Difference between AlexNet, VGGNet, ResNet, and
Inception. Towards Data Science.
https://towardsdatascience.com/the-w3h-of-alexnet-vggnet-resnet-and-inception-7b
aaaecccc96.
[70] Wuttke, L. (2022, April 27). Machine Learning vs. Deep Learning: Wo ist der Unterschied?
Datasolut GmbH. https://datasolut.com/machine-learning-vs-deep-learning/.
[71] The MathWorks, Inc. (n.d.). Texture Analysis Using the Gray-Level Co-Occurrence Matrix
(GLCM) - MATLAB & Simulink - MathWorks Deutschland. MathWorks® Help Center.
Retrieved January 29, 2023, from
https://de.mathworks.com/help/images/texture-analysis-using-the-gray-level-co-occ
urrence-matrix-glcm.html.
156

[72] The MathWorks, Inc. (n.d.). Texture Analysis - MATLAB & Simulink - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 29, 2023, from
https://de.mathworks.com/help/images/texture-analysis-1.html?s_tid=CRUX_lnav.
[73] Fernando, J. (2022, December 19). R-Squared formula, regression, and interpretations.
Investopedia. Retrieved January 29, 2023, from
https://www.investopedia.com/terms/r/r-squared.asp.
157

Pictorial Sources
Fig. 1: Webcam image from Norderney, taken on 8th August 2015 at 10:10 UTC+2.
Keuschnig, G., & Radlherr, F. (2015, August 8). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 29,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/08/08/1010.
Fig. 2:Beach on Wangerooge aer hurricane Zeynep.
Hannoversche Allgemeine Zeitung. (2022, February 19). Nordsee-Inseln: Sturm “Zeynep”
spült Strände fast komplett weg. Hannoversche Allgemeine Zeitung. Retrieved
January 29, 2023, from
https://www.haz.de/der-norden/nordsee-inseln-sturm-zeynep-spuelt-straende-fast-k
omplett-weg-7NYUVODMNRC4NPXRDA6JPAQYKI.html.
Fig. 3:East Frisian Islands offshore of the German North Sea coast.
Dörrbecker, M. (2018). Ostfriesische Inseln (Karte). Retrieved January 17, 2023, from
https://commons.wikimedia.org/wiki/File:Ostfriesische_Inseln_(Karte).png.
Fig. 4:Groyne field before the nourishment, 18 June 2019 at 9:00 UTC+2.
Keuschnig, G., & Radlherr, F. (2019, June 18). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 17,
2023, from https://www.foto-webcam.eu/webcam/norderney/2019/06/18/0900.
Fig. 5:Groyne field during the nourishment, 23 July 2019 at 9:00 UTC+2.
Keuschnig, G., & Radlherr, F. (2019, July 23). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 17,
2023, from https://www.foto-webcam.eu/webcam/norderney/2019/07/23/0900.
Fig. 6:Groyne field aer the nourishment, 27 July 2019 at 9:00 UTC+2.
Keuschnig, G., & Radlherr, F. (2019, July 27). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 17,
2023, from https://www.foto-webcam.eu/webcam/norderney/2019/07/27/0900.
Fig. 7:Groyne field with placements of combined (1992) and conventional (1989) nourishments
on Norderney. Niemeyer et al., 1997.
158

Fig. 8:Webcam position on Norderney.
Keuschnig, G., & Radlherr, F. (n.d.). Kaiserhof - Norderney / Ostfriesland - Blick Richtung
Nordwesten - Foto-Webcam.eu. Umgebungskarte Norderney. Foto-Webcam.Eu.
Retrieved January 10, 2023, from
https://www.foto-webcam.eu/webcam/norderney/2015/08/15/1450.
Fig. 9:View of the groyne field from the webcam, 18 July 2015 at 9:00 UTC+2.
Keuschnig, G., & Radlherr, F. (2015, July 18). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 17,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/07/18/0900.
Fig. 10:Webcam image with the analysis area of the sky framed in red (Reuter, 2022).
Fig. 11:Transformation stages of the cropped image of the groyne (Reuter, 2022).
Fig. 12:Images from the 4th May 2016, from the 4th June 2016, and their superimposed image
(Reuter, 2022).
Fig. 13:Collage of water lines from the 8th to the 14th January 2015 (Reuter, 2022).
Fig. 14:Tide curves of measured and calculated water levels (Reuter, 2022).
Fig. 15:Scatter plots for water levels and wave heights (Reuter, 2022).
Fig. 16:Original and oversegmented image. Hoonhout et al. (2015).
Fig. 17:Original image and transformed image with ROI. Hies et al. (2012).
Fig. 18:Images from Farson river camera. Vandaele et al. (2021).
Fig. 19:Images taken by an ASV at different sea state levels. Wang et al. (2013).
Fig. 20:Camera angle of vision with positions of the camera, the wave buoy (Boje), and the
water gauge (Pegel). Reuter, 2022.
Fig. 21:Image taken by the Norderney webcam on 8th August 2015 at 08:50 UTC+2.
Keuschnig, G., & Radlherr, F. (2015, August 20). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 13,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/08/20/0850.
Fig. 22:Webcam snapshot from the 18th February 2012 at 12:30 UTC+1.
Keuschnig, G., & Radlherr, F. (2012, February 18). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 13,
2023, from https://www.foto-webcam.eu/webcam/norderney/2012/02/18/1230.
159

Fig. 23:Webcam snapshot from the 15th September 2017 at 12:10 UTC+2.
Keuschnig, G., & Radlherr, F. (2017, September 15). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 13,
2023, from https://www.foto-webcam.eu/webcam/norderney/2017/09/15/1210.
Fig. 24:Tide gauge Norderney-Riffgat. Hartmann, E. (n.d.). Norderney Pegelmesser.
Hartmann, E. (n.d.). Norderney Pegelmesser. Deutsche Leuchtfeuer. Retrieved January 25,
2023, from
https://www.deutsche-leuchtfeuer.de/nordsee/norderney-pegelmesser.html.
Fig. 29:Webcam snapshot with the groyne framed in red.
Keuschnig, G., & Radlherr, F. (2015, August 2). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 26,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/08/02/0630.
Fig. 30:Webcam image with ROIs for brightness detection.
Keuschnig, G., & Radlherr, F. (2015, January 1). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 26,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/01/01/1120.
Fig. 31:ROIs for fog and snow detection.
Keuschnig, G., & Radlherr, F. (2015, January 26). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 26,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/01/26/1100.
Keuschnig, G., & Radlherr, F. (2015, January 24). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 26,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/01/24/1120.
Fig. 33:Webcam image from 1. January 2015 at 13:30 UTC+1 and image section cropped to the
groyne.
Keuschnig, G., & Radlherr, F. (2015, January 1). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 26,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/01/01/1330.
160

Fig. 42:Relative prediction speeds and accuracies of different pretrained networks.
The MathWorks, Inc. (n.d.). Pretrained Deep Neural Networks - MATLAB & Simulink - MathWorks
Deutschland. MathWorks® Help Center. Retrieved January 27, 2023, from
https://de.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-n
etworks.html.
Fig. 45:Network architecture for a model with two inputs.
The MathWorks, Inc. (n.d.). Train Network on Image and Feature Data - MATLAB & Simulink -
MathWorks Deutschland. MathWorks® Help Center. Retrieved January 27, 2023, from
https://de.mathworks.com/help/deeplearning/ug/train-network-on-image-and-featu
re-data.html.
Fig. 98:Daylight image falsely sorted into the twilight category, taken on 4th February
2015,10:30 UTC+1.
Keuschnig, G., & Radlherr, F. (2015, February 4). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 29,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/02/04/1030.
Fig. 99:Daylight image with cloudy skies, falsely sorted into the twilight category, taken on
15th June 2015, 10:00 UTC+2.
Keuschnig, G., & Radlherr, F. (2015, June 15). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 29,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/06/15/1000.
Fig. 100:Night image wrongly classified as daylight image and cropped, taken on 27th
January 2015 at 18:00 UTC+1.
Keuschnig, G., & Radlherr, F. (2015, January 27). Kaiserhof - Norderney / Ostfriesland - Blick
Richtung Nordwesten - Foto-Webcam.eu. Foto-Webcam.Eu. Retrieved January 29,
2023, from https://www.foto-webcam.eu/webcam/norderney/2015/01/27/1800.
Fig. 106:Scatter plot for water levels from 2015 and 2016 (Reuter, 2022).
161

Appendices
Nr.
Name
I
Preprocessing with Interpolation
II
Regression with Interpolation
III
PC Specifications and MATLAB® Toolboxes
IV
Edits and Additional Notes
Models and scripts were compressed into the file MA_JuliaSteiwer_autoAnalysis.zip
and uploaded to the directory WebcamNorderney/Matlab of the LUH-Cloud Seafile folder
belonging to Dr.-Ing. Jan Visscher.
The zip file contains the following files:
- trained regression models and corresponding scripts
- preprocessing routine, with and without interpolation
- regression routine, with and without interpolation
- prediction and training data
- ResNet18 for classification with difference images
- deep learning regression models
- modified ResNet18
- double-input network
162

Pre-Processing for Regression Task
The following script aims to pre-process image data for a regression task.
By analysing the mean gray-value of the sky, ocean, and ground, the images are sorted into three
categories of light conditions, namely "daylight", "twilight", and "night". The images that fall into the
"daylight" categories are then checked for the range of the gray-values of the sky and the mean of
the gray-values of the ground to detect whether fog or snow might be present. The images in which
neither was detected are then cropped to the ROI of the groyne.
A linking table with the image names, timestamps, water levels, wave heights, light conditions, and
beach visibility is then created.
The wave data that is being used here contains interpolated values; if you'd prefer using only
measured data, use the script preProcessing_withoutInterp.mlx instead.
Basics
Applies if the working directory is a folder named "Matlab". The parent folder of Matlab needs to
include the folder with the data (e.g. water levels and wave heights) as well as the folder with the
images. Change the names of the image and data folders depending on the names you have used.
clc
clear
close all
wd = pwd;
imgfolder = [wd(1:end-6) 'Bilder2015\']; % folder with image data
datafolder = [wd(1:end-6) '\Daten\']; % folder with data
piclist = dir([imgfolder '*hu.jpg']);
Timestamp
Get the timestamp "zst" from the image names.
zst = NaT(length(piclist), 1);
for i = 1:length(piclist)
zst(i,:) = datetime(piclist(i).name(11:21),'InputFormat','yyMMdd-HHmm');
end
Water Level
Get the water levels for each timestamp.
load('allewstand.mat')

wstand = zeros(size(piclist));
for i=1:length(piclist)
wstand(i,1) = wst(dattim == zst(i)); % might cause issues if there are
large data gaps
end
clear wst dattim
Wave Height
Get the wave height Hs and save it in the variable "welleHs". If the usage of the steepness Ss is
necessary, use the same code but replace welleHs with welleSs.
load('allewelle.mat')
[Lia,Locb] = ismember(zst, dattimwelle); % might have gaps if there are data
gaps
welleHs = NaN(size(piclist));
welleHs(Lia) = Hs(Locb(Lia));
welleHs = interp1(zst(Lia), welleHs(Lia), zst); % gaps are filled here using
interpolation
Visualize Data
Visualize the wave height and water level data to detect potential gaps or phase shifts, which might
produce problems in the analysis.
Wave Height
Plot the measured wave height data welleHs(Lia) against the interpolated wave data welleHs to
determine if data gaps exist.
plot(zst(Lia), welleHs(Lia), '.', "MarkerSize", 20);
hold on
plot(zst, welleHs);
title("Measured vs Interpolated Wave Height (2015)");
xlabel("Measurement Time Point");
ylabel("Wave Height in cm");
legend("Measured Wave Height", "Linearly Interpolated Wave height");
hold off
Water Level
Plot the water level wstand, which was calculated from wst based on the image timestamp zst,
against the timestamp to determine if data gaps exist.

plot(zst, wstand, '.', "MarkerSize", 5);
title("Measured Water Levels (2015)");
xlabel("Measurement Time Point");
ylabel("Water Level in cm");
Remove Data Gaps
Visualize the data using the plots from the previous section. Note down the timestamps for which
there are larger data gaps.
The wave height data from 2015 contains three major data gaps:
• 5th May 2015, 9:30 UTC+2 until 4th June 2015, 12:00 UTC+2,
• 5th October 2015, 13:40 UTC+2 until 14th October 2015, 13:30, and
• 24th November 2015, 21:00 UTC+1 until 8th December 2015, 7:30 UTC+1.
Find the row numbers of these entries in the timestamp vector zst, then use these numbers to
remove the images taken during the data gaps from the image list piclist. For the aforementioned
gaps, this would be:
idx1 = find(zst == datetime('05-May-2015 09:30:00'));
idx2 = find(zst == datetime('04-Jun-2015 12:00:00'));
idx3 = find(zst == datetime('05-Oct-2015 13:40:00'));
idx4 = find(zst == datetime('14-Oct-2015 13:30:00'));
idx5 = find(zst == datetime('24-Nov-2015 21:00:00'));
idx6 = find(zst == datetime('08-Dec-2015 07:30:00'));
The indexes can now be used to remove the images taken during the data gaps from the image list
piclist.
piclist = piclist([1:idx1, idx2:idx3, idx4:idx5, idx6:end]);
Update Data
Timestamps
zst = NaT(length(piclist), 1);
for i = 1:length(piclist)
zst(i,:) = datetime(piclist(i).name(11:21),'InputFormat','yyMMdd-HHmm');
end
Water Level
wstand = zeros(size(piclist));

for i=1:length(piclist)
wstand(i,1) = wst(dattim == zst(i)); % might cause issues if there are
large data gaps
end
clear wst dattim
Wave Height
[Lia,Locb] = ismember(zst, dattimwelle);
welleHs = NaN(size(piclist));
welleHs(Lia) = Hs(Locb(Lia));
welleHs = interp1(zst(Lia), welleHs(Lia), zst);
Create Sub-Folders
Append the path of the main image folder with paths to the new sub-folders.
Night = strcat(fileparts(imgfolder),'\Night');
Twilight = strcat(fileparts(imgfolder),'\Twilight');
badWeather = strcat(fileparts(imgfolder),'\Bad_Weather');
Cropped = strcat(fileparts(imgfolder),'\Cropped\'); % folder that will include
the cropped daylight images taken during good weather
Remove directories if they already exist. Commands are currently commented out as they would
cause problems when running the code for the first time without these directories already existing.
rmdir (Night,'s')
rmdir (Twilight,'s')
rmdir (badWeather,'s')
rmdir (Cropped,'s')
Make the new directories.
mkdir (Night)
mkdir (Twilight)
mkdir (badWeather)
mkdir (Cropped)
Define ROI
Define the ROI (region of interest, here the groyne) in the image.
The image used here is an image taken at very low tide (water level below 350 cm) and calm sea
(wave height below 10 cm) where the gryone was completely visible, which is necessary for
choosing an ROI that serves as a reference point for the entire dataset.

ind = find(wstand < 350 & welleHs < 10);
list = piclist(ind);
fldr = list(1).folder;
nm = list(1).name;
filename = strcat(fldr, '\', nm);
roi_img = imread(filename);
Display the image, then draw the ROI within the image and save its position in the new variable
"pos".
imshow(roi_img)
r1 = drawrectangle('Label','Groyne','Color',[1 0 0]);
pos = r1.Position;
Sort, Crop, and Save Daylight Images
Crop the images to the ROI defined above, reduce them to half their size (optional) and save them in
a new sub-folder named "Cropped".
Light = strings([length(piclist), 1]);
dispImg = imread([imgfolder piclist(1).name]);
imshow(dispImg)
for i=1:length(piclist) % starts from 1
IMG2 = imread([imgfolder piclist(i).name]);
GrayIMG = rgb2gray(IMG2);
%% Make Masks for Detection
% -- (positions have been determined previously and might need to be
changed depending on the field of view)
% Rectangle within the Ocean
h = drawrectangle('Position',[4120, 1125, 150, 150],'StripeColor','r');
msk = createMask(h);
% Rectangle within the Ground Area
g = drawrectangle('Position', [3845, 2275, 425, 275], 'StripeColor', 'r');
msk2 = createMask(g);
%% Get the Gray-Value Means for each ROI
meanSky = mean(GrayIMG(200:600)); % image rows below the logo + labels,
slightly above the horizon
meanOcean = mean(GrayIMG(msk)); % ROI within the ocean, slight to the top-
right of the groyne

meanGround = mean(GrayIMG(msk2)); % ROI of the ground, at the bottom-right
of the image
% Snow Detection
vertices = [2 1436; 2 2540; 4270 2564; 4270 2132];
p = drawpolygon('Position', vertices, 'Label', 'Ground');
msk3 = createMask(p);
meanGround2 = mean(GrayIMG(msk3));
% Fog Detection
vertices2 = [2 83; 2 1330; 4270 1910; 4270 245];
p2 = drawpolygon('Position', vertices2, 'Label', 'Sky + Groyne');
msk4 = createMask(p2);
rangeSky = range(GrayIMG(msk4));
%% Sort Images into Folders and write names of Light Conditions into
variable "Light"
% Limits were determined empirically and might need to be changed
% depending on the observation site.
if meanSky < 100 || meanOcean < 50 || meanGround < 15
copyfile ([imgfolder piclist(i).name], Night)
Light(i,:) = ("Night");
elseif meanSky < 150 || meanOcean < 65 || meanGround < 30
copyfile ([imgfolder piclist(i).name], Twilight)
Light(i,:) = ("Twilight");
else
if meanGround >= 125 || rangeSky < 200
copyfile ([imgfolder piclist(i).name], badWeather)
else
imgCrop = imcrop(IMG2, pos); % crop image using predefined ROI
imgCrop = imresize(imgCrop, 0.5); % decrease image size (optional)
filename = strcat(Cropped, piclist(i).name);
imwrite(imgCrop, filename);
end
Light(i,:) = ("Day");
end
end
Linking Table
Create the linking table from image names, timestamps, water levels, wave heights, and light
conditions.

imgtable = struct2cell(piclist)';
linkingTable = [strcat(imgtable(:,2), '\', imgtable(:,1)), string(zst), wstand,
welleHs, Light];
Beach Visibility
Check whether the beach is visible in the images and save the results in the linking table.
Beach = strings([length(piclist),1]);
Beach(:) = "no beach visible";
for beachCounter = 1:length(linkingTable)
if double(linkingTable(beachCounter,3)) < 500 % beach is considered visible
for a water level below 500 cm
Beach(beachCounter,:)= ("beach visible");
end
end

Regression of Wave Heights and Water Levels
The following script serves the purpose of calculating the wave heights and water levels of images
from their feature data.
The wave data that is being used here contains interpolated values; if you'd prefer using only
measured data, use the script regression_withoutInterp.mlx instead.
Basics
Applies if the working directory is a folder named "Matlab". The parent folder of Matlab needs to
include the folder with the data (e.g. water levels and wave heights) as well as the folder with the
images. Change the names of the image and data folders depending on the names you have used.
clc
clear
close all
wd = pwd;
imgfolder = [wd(1:end-6) 'Bilder2015\Cropped\'];
datafolder = [wd(1:end-6) '\Daten\'];
piclist = dir([imgfolder '*hu.jpg']);
sz = numel(piclist);
Regression Data for Wave Height
Timestamps
Get the timestamps of all images to assign the measured wave heights accordingly.
for i = 1:sz
% Timestamp
zst(i,:) = datetime(piclist(i).name(11:21),'InputFormat','yyMMdd-HHmm');
end
Water Level
Get the water levels for each timestamp.
load('allewstand.mat')
wstand = zeros(size(piclist));
for i=1:length(piclist)
wstand(i,1) = wst(dattim == zst(i)); % might cause issues if there are
large data gaps
end

clear wst dattim
Wave Height
Get the wave height Hs and save it in the variable "welleHs". If the usage of the steepness Ss is
necessary, use the same code but replace welleHs with welleSs.
load('allewelle.mat')
[Lia,Locb] = ismember(zst, dattimwelle); % might have gaps if there are data
gaps
welleHs = NaN(size(piclist));
welleHs(Lia) = Hs(Locb(Lia));
welleHs = interp1(zst(Lia), welleHs(Lia), zst); % gaps are filled here using
interpolation
Visualize Data
Visualize the wave height and water level data to detect potential gaps or phase shifts, which might
produce problems in the analysis.
Wave Height
Plot the measured wave height data welleHs(Lia) against the interpolated wave data welleHs to
determine if data gaps exist.
plot(zst(Lia), welleHs(Lia), '.', "MarkerSize", 20);
hold on
plot(zst, welleHs);
title("Measured vs Interpolated Wave Height (2015)");
xlabel("Measurement Time Point");
ylabel("Wave Height in cm");
legend("Measured Wave Height", "Linearly Interpolated Wave height");
hold off
Water Level
Plot the water level wstand, which was calculated from wst based on the image timestamp zst,
against the timestamp to determine if data gaps exist.
plot(zst, wstand, '.', "MarkerSize", 5);
title("Measured Water Levels (2015)");
xlabel("Measurement Time Point");
ylabel("Water Level in cm");

Find Data Gaps
Using the section "Remove Data Gaps" from preProcessing_withInterp.mlx, the row numbers of
the start and end points of data gaps can be used to index into piclist and remove the images
taken during larger gaps.
First, create indexes based on the row numbers of the gap timestamps.
idx1 = find(zst == datetime('05-May-2015 09:30:00')); % 1st gap start
idx2 = find(zst == datetime('04-Jun-2015 12:00:00')); % 1st gap end
idx3 = find(zst == datetime('05-Oct-2015 13:50:00')); % 2nd gap start
idx4 = find(zst == datetime('14-Oct-2015 13:20:00')); % 2nd gap end
idx5 = find(zst == datetime('24-Nov-2015 16:50:00')); % 3rd gap start
idx6 = find(zst == datetime('08-Dec-2015 08:00:00')); % 3rd gap end
Use the indices to remove the gap images from piclist and save the result back to to piclist.
piclist = piclist([1:idx1, idx2:idx3, idx4:idx5, idx6:end]);
Update size indicator sz:
sz = numel(piclist);
Arrays for Image Features
Create empty arrays for the variance, range, and mean of the GLCM (gray-level co-occurence
matrix) as well as the properties of the GLCM (graycoprops).
% Variance
variance_ss = zeros(sz, 8);
% Mean
mean_ss = zeros(sz, 8);
% Range
range_ss = zeros(sz, 8);
% GLCM Properties
prop_ss = zeros(sz, 4);
Update Timestamps
zst = NaT(sz, 1);
for i = 1:sz
zst(i,:) = datetime(piclist(i).name(11:21),'InputFormat','yyMMdd-HHmm');
end

Wave Height
Get the measured wave heights for each timestamp of an image and interpolate the data if there is
no match.
load('allewelle.mat')
[Lia,Locb] = ismember(zst, dattimwelle);
welleHs = NaN(sz);
welleHs(Lia) = Hs(Locb(Lia));
welleHs = interp1(zst(Lia), welleHs(Lia), zst);
Regression Data Array
Create an array of the regression data that matches the features of each image to the corresponding
measured or interpolated wave height.
for i = 1:sz
img = imread([imgfolder piclist(i).name]); % read image to workspace
img = rgb2gray(img); % convert to grayscale
% GLCM Feature Extraction
GLCM = graycomatrix(img);
props = graycoprops(GLCM);
% Variance
variance_ss(i, 1:8) = var(GLCM);
% Mean
mean_ss(i, 1:8) = mean(GLCM);
% Range
range_ss(i, 1:8) = range(GLCM);
% Properties of the GLCM
prop_ss(i, 1) = props.Contrast;
prop_ss(i, 2) = props.Correlation;
prop_ss(i, 3) = props.Energy;
prop_ss(i, 4) = props.Homogeneity;
%% Make Arrays of Regression Data
% Training Data (Features + Wave Height)
tableProps = [variance_ss, mean_ss, range_ss, prop_ss, welleHs];
% Prediction Data
predData = [variance_ss, mean_ss, range_ss, prop_ss];
end
Wave Height Prediction

Predict the wave height from the image features.
Load the trained regression model trainedModel.mat.
load('trainedModel.mat')
If you would like to retrain the model on the image features and measured wave heights, uncomment
the code below.
%[trainedModel, validationRMSE] = trainRegressionModel(tableProps);
Use the trained model and the image features (prediction data) to calculate the wave heights.
yfit = trainedModel.predictFcn(predData);
Water Level Prediction
Feature Data for Water Level Prediction
Set the limits for the wave heights to distinguish between the different sea states (refer to WMO sea
state code).
s1 = 10; % calm sea, max. 10cm wave height
s2 = 50; % smooth sea, max. 50cm wave height
s3 = 125; % slight sea, max. 125cm wave height
s4 = 250; % moderate sea, max. 250cm wave height
Get Feature Data Depending on the Measured Wave Height
Get the measured water level for each timestamp.
load('allewstand.mat')
wstand = zeros(size(piclist));
for i=1:length(piclist)
wstand(i,1) = wst(dattim == zst(i)); % might cause issues if there are
large data gaps
end
clear wst dattim
Make an array with the image features, water levels, and wave heights.
B = [predData wstand]; % combine image features and measured water level
C = [B welleHs]; % combine B and measured wave height
Find entries in the feature data that match the wave height in welleHs according to the sea states.

ind_clm = find(C(:, 30) <= s1);
ind_smth = find(C(:, 30) <= s2 & C(:, 30) > s1);
ind_slght = find(C(:, 30) <= s3 & C(:, 30) > s2);
ind_mdrt = find(C(:, 30) <= s4 & C(:, 30) > s3);
Get the prediction data (image features) for the water levels depending on sea state.
predCalm = predData(ind_clm, :);
predSmooth = predData(ind_smth, :);
predSlight = predData(ind_slght, :);
predModerate = predData(ind_mdrt, :);
Make training data for the water levels depending on sea state (includes the image features and the
measured water levels).
RD_calm = B(ind_clm, :);
RD_smooth = B(ind_smth, :);
RD_slight = B(ind_slght, :);
RD_mod = B(ind_mdrt, :);
Get Feature Data Depending on the Predicted Wave Height
Create an array with the image features and the predicted wave heights.
A = [predData wstand yfit];
Find entries in the feature data that match the wave height in yfit according to the sea states.
ind_calm = find(A(:, 30) <= s1);
ind_smooth = find(A(:, 30) <= s2 & A(:, 30) > s1);
ind_slight = find(A(:, 30) <= s3 & A(:, 30) > s2);
ind_mod = find(A(:, 30) <= s4 & A(:, 30) > s3);
Split the feature data into the four sea state catgeories and save them in new variables.
predData_calm = predData(ind_calm, :);
predData_smooth = predData(ind_smooth, :);
predData_slight = predData(ind_slight, :);
predData_mod = predData(ind_mod, :);
Predict Water Levels
Load the trained regression models.
load('trainedModelCalm.mat')
load('trainedModelSmooth.mat')
load('trainedModelSlight.mat')

load('trainedModelModerate.mat')
If you would like to retrain the models on the image features and measured water levels depending
on sea state, uncomment the code below.
%[trainedModelCalm, validationRMSEcalm] = trainRegressionModelCalm(RD_calm);
%[trainedModelSmooth, validationRMSEsmooth] =
trainRegressionModelSmooth(RD_smooth);
%[trainedModelSlight, validationRMSEslight] =
trainRegressionModelSlight(RD_slight);
%[trainedModelModerate, validationRMSEmod] =
trainRegressionModelModerate(RD_mod);
Use the trained models and image features to predict the water levels. The inputs starting with
predData_ were derived from the predicted wave height yfit. If you would like to use inputs that
were derived from the measured wave height welleHs, replace the predData_ inputs as detailed in
the comments.
yfit_calm = trainedModelCalm.predictFcn(predData_calm); % for welleHs: predCalm
yfit_smooth = trainedModelSmooth.predictFcn(predData_smooth); % for welleHs:
predSmooth
yfit_slight = trainedModelSlight.predictFcn(predData_slight); % for welleHs:
predSlight
yfit_moderate = trainedModelModerate.predictFcn(predData_mod); % for welleHs:
predModerate
Visualize Results with Histograms
Compare the measured vs. predicted results visually. The bin-width can be changed to make
differences or small numbers of observations more visible, however, the bin-width needs to have the
same value for both histograms.
Predicted vs. Measured Wave Height
Plot the histogram of the measured wave heights, then overlay it with the histogram of predicted
wave heights.
h1 = histogram(welleHs, 'BinWidth', 5);
title("Histogram of Predicted vs. Measured Wave Height (Sea State, 2015)");
ylabel("Number of Observations");
xlabel("Wave Height in cm");
hold on
h2 = histogram(yfit, 'BinWidth', 5);
legend("Measured Wave Height", "Predicted Wave Height");
hold off

Predicted vs. Measured Water Level
Calm Sea
Plot the histogram of the measured water levels during calm seas, then overlay it with the histogram
of predicted water levels during calm seas. If you used the features that depend on the measured
water level, replace ind_calm with ind_clm.
h1_calm = histogram(wstand(ind_calm, :), 'BinWidth', 50);
title("Histogram of Predicted vs. Measured Water Level (Calm Sea, 2015)");
ylabel("Number of Observations");
xlabel("Water Level in cm");
hold on
h2_calm = histogram(yfit_calm, 'BinWidth', 50);
legend("Measured Water Level", "Predicted Water Level");
hold off
Smooth Sea
Plot the histogram of the measured water levels during smooth seas, then overlay it with the
histogram of predicted water levels during smooth seas. If you used the features that depend on the
measured water level, replace ind_smooth with ind_smth.
h1_smooth = histogram(wstand(ind_smooth, :), 'BinWidth', 5);
title("Histogram of Predicted vs. Measured Water Level (Smooth Sea, 2015)");
ylabel("Number of Observations");
xlabel("Water Level in cm");
hold on
h2_smooth = histogram(yfit_smooth, 'BinWidth', 5);
legend("Measured Water Level", "Predicted Water Level");
hold off
Slight Sea
Plot the histogram of the measured water levels during slight seas, then overlay it with the histogram
of predicted water levels during slight seas. If you used the features that depend on the measured
water level, replace ind_slight with ind_slght.
h1_slight = histogram(wstand(ind_slight, :), 'BinWidth', 5);
title("Histogram of Predicted vs. Measured Water Level (Slight Sea, 2015)");
ylabel("Number of Observations");
xlabel("Water Level in cm");
hold on
h2_smooth = histogram(yfit_slight, 'BinWidth', 5);
legend("Measured Water Level", "Predicted Water Level");

hold off
Moderate Sea
Plot the histogram of the measured water levels during moderate seas, then overlay it with the
histogram of predicted water levels during moderate seas. If you used the features that depend on
the measured water level, replace ind_mod with ind_mdrt.
h1_mod = histogram(wstand(ind_mod, :), 'BinWidth', 50);
title("Histogram of Predicted vs. Measured Water Level (Moderate Sea, 2015)");
ylabel("Number of Observations");
xlabel("Water Level in cm");
hold on
h2_smooth = histogram(yfit_moderate, 'BinWidth', 50);
legend("Measured Water Level", "Predicted Water Level");
hold off
Visualize Results by Plotting Predicted against Measured Values
Show the results of the measured vs. predicted values visually. The line y = x indicates the perfect
prediction of the measured values.
Predicted vs. Measured Wave Height
Plot the predicted wave height against the measured wave height.
plot(welleHs, yfit, '.')
title("Predicted vs. Measured Wave Height (2015)");
xlabel("Measured Wave Height [cm]");
ylabel("Predicted Wave Height [cm]");
x = [min(welleHs) max(welleHs)];
y = x;
hold on
plot(x, y, "LineWidth", 1, "Color", [0 0 0])
xlim([min(welleHs) max(welleHs)])
ylim([min(welleHs) max(welleHs)])
legend("Observations", "Perfect Prediction", "Location", "southeast");
hold off
Predicted vs. Measured Water Level
Calm Sea
Plot the predicted water levels at calm sea against the measured water levels. If you used the
features that depend on the measured water level, replace ind_calm with ind_clm.

plot(wstand(ind_calm, :), yfit_calm, '.')
title("Predicted vs. Measured Water Level (Calm Sea, 2015)");
xlabel("Measured Water Level [cm]");
ylabel("Predicted Water Level [cm]");
x = [min(wstand(ind_calm, :)) max(wstand(ind_calm, :))];
y = x;
hold on
plot(x, y, "LineWidth", 1, "Color", [0 0 0])
xlim([min(wstand(ind_calm, :)) max(wstand(ind_calm, :))])
ylim([min(wstand(ind_calm, :)) max(wstand(ind_calm, :))])
legend("Observations", "Perfect Prediction", "Location", "southeast");
hold off
Smooth Sea
Plot the predicted water levels at smooth sea against the measured water levels. If you used the
features that depend on the measured water level, replace ind_smooth with ind_smth.
plot(wstand(ind_smooth, :), yfit_smooth, '.')
title("Predicted vs. Measured Water Level (Smooth Sea, 2015)");
xlabel("Measured Water Level [cm]");
ylabel("Predicted Water Level [cm]");
x = [min(wstand(ind_smooth, :)) max(wstand(ind_smooth, :))];
y = x;
hold on
plot(x, y, "LineWidth", 1, "Color", [0 0 0])
xlim([min(wstand(ind_smooth, :)) max(wstand(ind_smooth, :))])
ylim([min(wstand(ind_smooth, :)) max(wstand(ind_smooth, :))])
legend("Observations", "Perfect Prediction", "Location", "southeast");
hold off
Slight Sea
Plot the predicted water levels at slight sea against the measured water levels. If you used the
features that depend on the measured water level, replace ind_slight with ind_slght.
plot(wstand(ind_slight, :), yfit_slight, '.')
title("Predicted vs. Measured Water Level (Slight Sea, 2015)");
xlabel("Measured Water Level [cm]");
ylabel("Predicted Water Level [cm]");
x = [min(wstand(ind_slight, :)) max(wstand(ind_slight, :))];
y = x;

hold on
plot(x, y, "LineWidth", 1, "Color", [0 0 0])
xlim([min(wstand(ind_slight, :)) max(wstand(ind_slight, :))])
ylim([min(wstand(ind_slight, :)) max(wstand(ind_slight, :))])
legend("Observations", "Perfect Prediction", "Location", "southeast");
hold off
Moderate Sea
Plot the predicted water levels at moderate sea against the measured water levels. If you used the
features that depend on the measured water level, replace ind_mod with ind_mdrt.
plot(wstand(ind_mod, :), yfit_moderate, '.')
title("Predicted vs. Measured Water Level (Moderate Sea, 2015)");
xlabel("Measured Water Level [cm]");
ylabel("Predicted Water Level [cm]");
x = [min(wstand(ind_mod, :)) max(wstand(ind_mod, :))];
y = x;
hold on
plot(x, y, "LineWidth", 1, "Color", [0 0 0])
xlim([min(wstand(ind_mod, :)) max(wstand(ind_mod, :))])
ylim([min(wstand(ind_mod, :)) max(wstand(ind_mod, :))])
legend("Observations", "Perfect Prediction", "Location", "southeast");
hold off
Plot Data over Time
Wave Height
Plot the measured (and interpolated) wave height data over time, and show where the predictions
place in comparison.
plot(zst, welleHs, '.', 'MarkerSize', 10);
hold on
plot(zst, yfit, '.', 'MarkerSize', 10);
title("Measured vs Predicted Wave Heights (2015)");
xlabel("Measurement Time Points");
ylabel("Wave Height in cm");
legend("Measured Wave Heights", "Predicted Wave Heights");
hold off
Water Level

Calm Sea
Plot the measured water level data for calm seas over time, and show where the predictions place in
comparison. If you used the features that depend on the measured water level, replace ind_calm
with ind_clm.
plot(zst(ind_calm), wstand(ind_calm), '.', 'MarkerSize', 10);
hold on
plot(zst(ind_calm), yfit_calm, '.', 'MarkerSize', 10);
title("Measured vs Predicted Water Levels (Calm Sea, 2015)");
xlabel("Measurement Time Points");
ylabel("Water Level in cm");
legend("Measured Water Levels", "Predicted Water Levels");
hold off
Smooth Sea
Plot the measured water level data for smooth seas over time, and show where the predictions place
in comparison. If you used the features that depend on the measured water level, replace
ind_smooth with ind_smth.
plot(zst(ind_smooth), wstand(ind_smooth), '.', 'MarkerSize', 10);
hold on
plot(zst(ind_smooth), yfit_smooth, '.', 'MarkerSize', 10);
title("Measured vs Predicted Water Levels (Smooth Sea, 2015)");
xlabel("Measurement Time Points");
ylabel("Water Level in cm");
legend("Measured Water Levels", "Predicted Water Levels");
hold off
Slight Sea
Plot the measured water level data for slight seas over time, and show where the predictions place
in comparison. If you used the features that depend on the measured water level, replace
ind_slight with ind_slght.
plot(zst(ind_slight), wstand(ind_slight), '.', 'MarkerSize', 10);
hold on
plot(zst(ind_slight), yfit_slight, '.', 'MarkerSize', 10);
title("Measured vs Predicted Water Levels (Slight Sea, 2015)");
xlabel("Measurement Time Points");
ylabel("Water Level in cm");
legend("Measured Water Levels", "Predicted Water Levels");

hold off
Moderate Sea
Plot the measured water level data for moderate seas over time, and show where the predictions
place in comparison. If you used the features that depend on the measured water level, replace
ind_mod with ind_mdrt.
plot(zst(ind_mod), wstand(ind_mod), '.', 'MarkerSize', 10);
hold on
plot(zst(ind_mod), yfit_moderate, '.', 'MarkerSize', 10);
title("Measured vs Predicted Water Levels (Moderate Sea, 2015)");
xlabel("Measurement Time Points");
ylabel("Water Level in cm");
legend("Measured Water Levels", "Predicted Water Levels");
hold off
Plot All Water Levels over Time
Make timetables for the water levels depending on different sea states.
% Calm Sea
yf = yfit_calm;
TCalm = timetable(zst(ind_calm), yf);
% Smooth Sea
yf = yfit_smooth;
TSmooth = timetable(zst(ind_smooth), yf);
% Slight Sea
yf = yfit_slight;
TSlight = timetable(zst(ind_slight), yf);
% Moderate Sea
yf = yfit_moderate;
TMod = timetable(zst(ind_mod), yf);
Combine the individual timetables into one large timetable and sort ascending by time.
TComb = vertcat(TCalm, TSmooth, TSlight, TMod);
TComb = sortrows(TComb,'Time','ascend');
Plot all of the predicted and the measured water levels over the whole time frame.

plot(zst, wstand, 'Marker', '.', 'MarkerSize', 10)
hold on
plot(zst, TComb.yf, '.', 'MarkerSize', 10)
title("Measured vs Predicted Water Levels (2015)");
xlabel("Measurement Time Points");
ylabel("Water Level in cm");
legend("Measured Water Levels", "Predicted Water Levels");
hold off
Plot a histogram with the distribution of the water levels.
h1_all = histogram(wstand, 'BinWidth', 5);
title("Histogram of Predicted vs. Measured Water Levels (2015)");
ylabel("Number of Observations");
xlabel("Water Level in cm");
hold on
h2_all = histogram(TComb.yf, 'BinWidth', 5);
legend("Measured Water Level", "Predicted Water Level");
hold off
Calculate RMSE
Calculate the RMSE for each predicted value, and the mean RMSE for all predictions.
Wave Height
sz = numel(yfit);
rmse = zeros(sz, 1);
for i = 1:sz
rmse(i,1) = round(sqrt(mean((welleHs(i) - yfit(i)).^2)));
end
rmse2 = rmmissing(rmse);
rmseTotal = round(mean(rmse2));
stdRMSE = round(std(rmse2));
Water Level
Calm Sea
If you used the features that depend on the measured water level, replace ind_calm with ind_clm.
sz_calm = numel(yfit_calm);
rmse_calm = zeros(sz, 1);

ws_calm = wstand(ind_calm);
for i = 1:sz_calm
rmse_calm(i,1) = round(sqrt(mean((ws_calm(i) - yfit_calm(i)).^2)));
end
rmse_calm = rmmissing(rmse_calm);
rmseTotal_calm = round(mean(rmse_calm));
stdRMSE_calm = round(std(rmse_calm));
Smooth Sea
If you used the features that depend on the measured water level, replace ind_smooth with
ind_smth.
sz_smooth = numel(yfit_smooth);
rmse_smooth = zeros(sz, 1);
ws_smooth = wstand(ind_smooth);
for i = 1:sz_smooth
rmse_smooth(i,1) = round(sqrt(mean((ws_smooth(i) - yfit_smooth(i)).^2)));
end
rmse_smooth = rmmissing(rmse_smooth);
rmseTotal_smooth = round(mean(rmse_smooth));
stdRMSE_smooth = round(std(rmse_smooth));
Slight Sea
If you used the features that depend on the measured water level, replace ind_slight with
ind_slght.
sz_slight = numel(yfit_slight);
rmse_slight = zeros(sz, 1);
ws_slight = wstand(ind_slight);
for i = 1:sz_slight
rmse_slight(i,1) = round(sqrt(mean((ws_slight(i) - yfit_slight(i)).^2)));
end
rmse_slight = rmmissing(rmse_slight);
rmseTotal_slight = round(mean(rmse_slight));
stdRMSE_slight = round(std(rmse_slight));
Moderate Sea
If you used the features that depend on the measured water level, replace ind_mod with ind_mdrt.

sz_mod = numel(yfit_moderate);
rmse_moderate = zeros(sz, 1);
ws_mod = wstand(ind_mod);
for i = 1:sz_mod
rmse_moderate(i,1) = round(sqrt(mean((ws_mod(i) - yfit_moderate(i)).^2)));
end
rmse_moderate = rmmissing(rmse_moderate);
rmseTotal_moderate = round(mean(rmse_moderate));
stdRMSE_mod = round(std(rmse_moderate));
Water Levels for All Sea States
sz_all = numel(TComb.yf);
rmse_wstand = zeros(sz, 1);
for i = 1:sz_all
rmse_wstand(i,1) = round(sqrt(mean((wstand(i) - TComb.yf(i)).^2)));
end
rmse_wstand = rmmissing(rmse_wstand);
rmseTotal_wstand = round(mean(rmse_wstand));
stdRMSE_wstand = round(std(rmse_wstand));

APPENDIX III
Appendix III
PC Specifications and MATLAB® Toolboxes
The image preprocessingand the feature extractionwas done on the Home PC, while the deep and
machine learning tasks were done on both the Home and the Remote PC.
The information about the devices, installed Windows versions as well as CPU and GPU was taken
from the “About this PC” section under “Settings” and the “Performance” tab of the Task Manager
(see Table A1.1 below).
Table A1.1: Home vs Remote PC Specifications
Specification
Home PC
Remote PC
Operating System
Microso Windows 11 Home
Version 10.0 (Build 22000)
Microso Windows 10 Pro
Version 10.0 (Build 19045)
CPU
Intel® Core™ i3-8100 CPU @
3.60 GHz
Intel® Core™ i7-6700 CPU @
3.40 GHz
- Cores
4
4
- Logical Processors
4
8
GPU
Intel® UHD Graphics 630
NVIDIA GeForce GTX 1060 6GB
- Supported by MATLAB
No
Yes [1]
Java Version
Java 1.8.0_202-b08 with
Oracle Corporation Java
HotSpot™ 64-Bit Server VM
mixed mode
Java 1.8.0_202-b08 with
Oracle Corporation Java
HotSpot™ 64-Bit Server VM
mixed mode
MATLAB Version
9.12.0.1975300 (R2022a)
Update 3
9.12.0.2039608 (R2022a)
Update 5
The Intel® GPU could unfortunately not be used for GPU computing, as MATLAB exclusively
supports NVIDIA(R) GPU architectures with compute capability 3.5 to 8.x. According to an answer in
the MATLAB Answers support forum by Walter Roberson, MathWorks relies on there being
third-party high performance scientific libraries for the target architectures, which are currently
provided by NVIDIA, IBM, and AMD but not by Intel [2].
Multiple toolboxes needed to be installed for the different tasks (preprocessing, feature extraction,
machine and deep learning) as well as for the parallel and GPU computing (see Table A1.2 below).
1

APPENDIX III
Table A1.2: Installed MATLAB Toolboxes.
Toolbox Name
Version
Computer Vision Toolbox
10.2 (R2022a)
Deep Learning Toolbox
14.4 (R2022a)
GPU Coder *
2.3 (R2022a)
Image Processing Toolbox
11.5 (R2022a)
MATLAB Coder **
5.4 (R2022a)
Parallel Computing Toolbox *
7.6 (R2022a)
Statistics and Machine Learning Toolbox
12.3 (R2022a)
System Identification Toolbox
9.16 (R2022a)
The toolboxes in Table 1 without an asterisk are needed for the deep and machine learning tasks,
either for using task specific commands or apps (e.g. Regression Learner App, Deep Network
Designer). A single asterisk (*) indicates that the toolbox is needed for performing parallel and GPU
computing. The double asterisk (**) as seen on the MATLAB Coder indicates that the toolbox was
installed but not necessary for either machine and deep learning or parallel and GPU computing.
[1] The MathWorks, Inc., “GPU Computing Requirements - MATLAB & Simulink - MathWorks
Deutschland,” MathWorks® Help Center.
https://de.mathworks.com/help/parallel-computing/gpu-computing-requirements.html (accessed
Jan. 12, 2023).
[2] W. Roberson, “Use Intel graphics card/driver for GPU programming on Matlab?,” MATLAB
Central, Jul. 18, 2021.
https://de.mathworks.com/matlabcentral/answers/881123-use-intel-graphics-card-driver-for-gpu-
programming-on-matlab#answer_749203 (accessed Jan. 12, 2023).
2

Appendix IV
Edits and Additional Notes
The published version of this thesis was corrected regarding false figure numbers and other small
errors (last edited: 8th April 2023). All corrections are listed below.
Page 18: “see Fig. 4 - 6 for a before, during and aer view, and Fig. 57for placements on the coast”
Page 25: “[…] and corresponding measured or interpolated water levels […]”
Page 26: “[…] at a water level of around 6,60 6.60 m […]”
Page 30: “Outliers are especially prevalent for high measured water levels (Fig. 615, le).”
Page 43: “As seen in Fig. 221, the camera faces the beach at an angle instead of head-on.”
Page 44: “[…] the angle of view differs notably from the one in Fig. 221 […]”
Page 45: “[…], the angle of view was changed once again (see Fig. 23)”
Page 50: “[…] all folders have to be within the same folder layer […]”
Page 51: “This name contains the timestamp in the format YYMMHHDD-hhmm […]”
Page 61: “This name contains the timestamp in the format YYMMHHDD-hhmm […]”
Page 62: “[…] (see figures 524,625,827, and 928 in 3.2 Data Sources).”
Page 67: “Using either the linking table created in 3.3.1 Preprocessing the Image Data […]”
Page 80: “In Fig. 142, […]”
Page 82: “[…] its layer graph is plotted in the le image of Fig. 28 47.”
Page 114: “The water level detection for calm moderate seas […]”
Page 130: “In the case of Fig. 93 99, […]”
Page 135: “Looking at the edge images in Fig. 98 104 […]”
Furthermore, it should be noted that instead of using the raw wave heights for the wave height
detection, the significant wave height HSwas used. However, HSis actually the average of the
highest one-third waves, meaning that HS does not actually reflect the wave height that is visible in
the webcam image. Therefore, it is strongly advised to retrain the here provided models on the raw
wave heights, as this would provide correct and improved results.