Access to this full-text is provided by MDPI.
Content available from Remote Sensing
This content is subject to copyright.
Citation: Huang, Q.; Jin, G.; Xiong,
X.; Ye, H.; Xie, Y. Monitoring Urban
Change in Conflict from the
Perspective of Optical and SAR
Satellites: The Case of Mariupol, a
City in the Conflict between RUS and
UKR. Remote Sens. 2023,15, 3096.
https://doi.org/10.3390/rs15123096
Academic Editors: Magaly Koch,
Guoqing Li, Feng Zhang and
Junshi Xia
Received: 4 May 2023
Revised: 5 June 2023
Accepted: 12 June 2023
Published: 13 June 2023
Copyright: © 2023 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
remote sensing
Article
Monitoring Urban Change in Conflict from the Perspective of
Optical and SAR Satellites: The Case of Mariupol, a City in the
Conflict between RUS and UKR
Qihao Huang 1, Guowang Jin 1, *, Xin Xiong 1, Hao Ye 1and Yuzhi Xie 2
1Institute of Geospatial Information, Information Engineering University, Zhengzhou 450001, China;
hqh_rs@163.com (Q.H.)
2School of Earth Sciences and Resources, China University of Geosciences (Beijing), Beijing 100083, China
*Correspondence: guowang_jin@163.com
Abstract:
Modern armed conflicts can cause serious humanitarian disasters, and remote sensing
technology is critical in monitoring war crimes and assessing post-war damage. In this study, a
constrained energy minimization algorithm incorporating the feature bands (IFB-CEM) is designed
to detect urban burning areas in optical images. Due to the difficulty of obtaining the ground survey
data of the battlefield, the dual-polarization normalized coherence index (DPNCI) is designed based
on the multi-temporal synthetic aperture radar (SAR) image, and the quantitative inversion and
evaluation of the destruction of urban architecture are combined with the public images on the
Internet. The results show that the burning area is widely distributed in the armed conflict region,
and the distribution is most concentrated around the Azovstal steel and iron works. The burning
area reached its peak around 22 March, and its change is consistent with the conflict process in time
and space. About 79.2% of the buildings in the city were severely damaged or completely destroyed,
and there was a significant correlation with burning exposure. The results of this study show that
publicly available medium-resolution remote sensing data and Internet information have the ability
to respond quickly to the damage assessment of armed conflict and can provide preliminary reference
information for dealing with humanitarian disasters.
Keywords:
Russia–Ukraine conflict; urban burning detection; synthetic aperture radar; coherence
change detection; damage assessment
1. Introduction
Modern armed conflicts will have a huge impact on the safety of human life and
property as well as the social and natural environment [
1
]. Since the 21st century, dozens
of armed conflicts have broken out in the world, including the war in Afghanistan, the
war in Iraq, the civil war in Syria, the civil war in Libya and the Russia–Ukraine conflict.
These conflicts not only changed the social, economic and political patterns of a country
but also subtly affected the trend of human civilization. The direct impact of armed conflict
is the loss of human lives and the destruction of infrastructure. What is more serious is that,
due to the lack of access to infrastructure and a safe living environment, conflict indirectly
causes mass migration of people and eventually turns into a series of serious humanitarian
disasters [2].
Unlike crises caused by natural disasters, the deterioration of regional security during
armed conflicts makes it difficult for international observers to conduct field investigations
into conflict areas and assess disaster losses. Low-altitude aircraft are also unable to operate
due to airspace security problems. Therefore, how to quickly obtain accurate temporal and
spatial dynamic information on humanitarian disasters, assess losses in disaster areas and
then guide the international community to mediate conflicts and carry out humanitarian
Remote Sens. 2023,15, 3096. https://doi.org/10.3390/rs15123096 https://www.mdpi.com/journal/remotesensing
Remote Sens. 2023,15, 3096 2 of 20
aid and post-disaster reconstruction has become an urgent problem for the international
community to solve in the face of armed conflict crisis.
The development of earth observation technology provides a better choice to solve this
problem. Remote sensing images can quickly obtain a wide range of surface information
and effectively detect the occurrence of malignant events and ground object damage in
humanitarian disaster areas. Therefore, some research institutions or researchers, includ-
ing the United Nations Institute for Training and Research (UNITAR), have conducted
humanitarian disaster monitoring and damage assessment in Iraq, Yemen, Syria and other
countries based on different types of remote sensing images [3–6].
In high-resolution optical remote sensing images (such as Worldview, Quickbird,
GeoEye), individual infrastructure damage and the occurrence of small crowd gatherings
can be detected. Marx et al. used Plant, a high-resolution satellite constellation with a
spatial resolution of 3 m, to detect the potentially burned villages [
7
] in the ethnic cleansing
campaign of Rohingya villages in northern Rakhine State by the Myanmar military in
2017 [
8
]. UNITAR evaluated the level of building damage in some areas during the
conflict between Russia and Ukraine through visual interpretation based on the Worldview
satellite with a resolution of 0.5 m [
9
]. However, the high cost and small imaging range of
high-resolution optical remote sensing images generally limits their application in large-
scale armed conflicts. Medium-resolution optical remote sensing images (such as Landsat
series, Sentinel-2 series and MODIS) have been widely used in large-scale armed conflict
monitoring due to their stable imaging quality, large observation perspective and short
revisit period. The main work is reflected in the detection of thermal anomaly areas in
conflicts and the detection of regional surface changes. For example, MODIS thermal
anomaly detection data products are applied to reveal the potential relationship between
fire growth and violent conflicts in Darfur, Sudan, proving that objective information on
some violent conflicts may be provided based on the global daily fire detection situation [
10
].
Landsat ETM+ was used to detect changes in the Darfur region of Sudan, and it was pointed
out that the cause of changes might be related to burning by comparing the albedo data of
the adjacent two years [
11
]. Marx modeled the expected daily surface reflectance of each
urban pixel in Aleppo and Damascus, Syria, based on the stability and radiative consistency
of the Landsat series of optical satellite constellations, combined with their historical archive
data, and evaluated the urban building damage in Syria during the conflict [
12
]. In addition,
nighttime light (NTL) can also reflect the impact of armed conflict on human social life to
some extent. In the early stage, the Defense Meteorological Satellite Program’s Operational
Linescan System (DMSP-OLS) was often used to monitor major events occurring in human
social life [
13
]. Li et al. applied the change of nighttime lighting area during the Syrian
crisis for a long time and concluded that there was a linear relationship between the loss
of nighttime light and the number of lost population, and confirmed the effectiveness of
NTL in monitoring humanitarian crisis [
14
]. In recent years, the Visible Infrared Imaging
Radiometer Suite (VIIRS) sensor on the National Polar-orbiting Partnership (NPP) satellite
has become a new source for monitoring light at night, and it can provide better quality
data [
15
], and these data were used in the Russia–Ukraine conflict in 2022 for war action
interpretation, socio-economic assessment and refugee population monitoring [16–18].
However, optical sensors are usually restricted by environmental factors such as
weather, climate and day and night. Especially during the conflict, smoke from burning
ground fuels severely blocks the propagation of visible light. Synthetic aperture radar
(SAR) provides a new perspective of observing the ground using a microwave. The
advantage of all-sky and all-weather makes up for the deficiency of optical sensors affected
by observation conditions [
19
]. Aimaiti, Y et al. used the change of SAR intensity and
texture analysis of optical images to detect the damage level of buildings in the Kiev area
during the Russia–Ukraine conflict, achieving a detection accuracy of 58% [
20
]. Washaya P
et al. used Sentinel-1 to monitor the coherent changes of natural and man-made disasters
occurring in Syria, Iran and other regions, and combined land use type data and coherent
map standard deviation to reveal the changes in building damage at street level [
21
]. In
Remote Sens. 2023,15, 3096 3 of 20
addition, Interferometric Synthetic Aperture Radar (InSAR) technology also provides a
more real-time and repeated continuous observation scheme for post-war urban destruction
mapping and damage assessment [22,23].
With the development of the Internet and social networks, more and more researchers
have noticed the importance of remote sensing images assisted by different sources and
multi-view images for earth observation [
24
]. Optical remote sensing images can be
combined with low-altitude aerial photography images and ground reconnaissance images,
which have been successful in monitoring the destruction of cultural relics and monuments
in parts of Iraq and Syria during the ISIS occupation [
25
]. The combination of time series
NTL and social media big data also has great potential in monitoring and understanding
crisis development and refugee flows because both are sensitive indicators of economic
and human capital loss, and big data and remote sensing data sets have the potential value
of providing classified and timely data for conflicts that lack official statistics [26].
In previous studies, the burning areas in armed conflict are usually based on the band
combination of multi-spectral satellites containing infrared bands and observed by human
eyes, which is highly subjective and tedious work. Moreover, the detection of ground object
damage changes caused by conflicts is usually unable to reach an accurate conclusion due to
the lack of effective and timely information references. Therefore, in this study, we designed
a semi-automated framework for detecting burning areas in conflict and building damage
changes by combining multi-source medium-resolution satellite data with open-source
Internet information. In this study, the city of Mariupol, which was severely damaged in
the conflict between Russia and Ukraine on 24 February 2022, was taken as an example. The
specific work was as follows: (1) The detection of urban burning areas during the battle of
Mariupol was carried out by using public Sentinel-2A/B and Landsat-8/9 satellite images;
(2) Temporal coherence image of Sentinel-1A satellite was used to deduce the distribution
and evolution of damaged buildings in conflicts with Open Street Map, social media data
and open data set, and the reliability of the method was tested by coherence changes of
burned buildings. (3) This paper comprehensively evaluates the damage situation of urban
buildings in Mariupol and discusses the advantages and limitations of using multi-source
medium resolution remote sensing satellites in the assessment of armed conflict damage.
2. Study Area and Data
2.1. Study Area
Located in the South of the Donetsk Prefecture and borders the Sea of Azov, it connects
the Donbas region to the Crimean Peninsula. Mariupol is the second-largest city in Donetsk
Prefecture and the tenth-largest city in Ukraine, the essential sea-land traffic control hub,
and an industrialized town for metallurgy, machinery and trade. Mariupol extends from
47
◦
1
0
26” N to 47
◦
13
0
19” N latitude and from 37
◦
27
0
41” E to 37
◦
47
0
17” E longitude, covering
an area of 244 km
2
. Topographically, it is part of the Sea of Azov lowland, without any
mountains or hills, and the elevation of the surrounding farmland is slightly higher than
that of the central city. The urban land is mainly concentrated at the mouth of the central
Karimius River, with good shipping resources, coupled with its historical coal mining, iron
and steel industry heritage, make it a strategic port city. As of the end of 2021, Mariupol
had a population of 431,859, with an urban area of 166 km
2
and a downtown area of
106 km2
[
27
]. Figure 1shows the geographical location of the study area and optical and
SAR images of the corresponding area.
Remote Sens. 2023,15, 3096 4 of 20
Figure 1. Geographical location of the study area.
On 24 February 2022, Russia broke out armed conflict with Ukraine on the grounds of
“demilitarization and de-Nazification” [
28
–
30
]. Because of its special geographical location
and important industrial resources, Mariupol became a battlefield in the conflict between
Russia and Ukraine. During the conflict, not only military facilities were severely destroyed,
but also civilian infrastructure, including urban residential areas, commercial centers and
religious sites. As of 16 May 2022, the entire territory of Mariupol is under the control of
Russian forces. Almost every building in Mariupol has been destroyed, and a large number
of people have been killed and injured (no specific statistics are available) during the 82-day
conflict, making the area the epicenter of a humanitarian disaster [31].
2.2. Data Source
The remote sensing data used in this study are all from freely available public satellite
images. The optical image mainly adopts the Sentinel-2A/2B satellite Level-1C images pro-
vided by European Space Agency (ESA) [
32
]. Sentinel-2 is a wide-swath, high-resolution,
multi-spectral imaging mission supporting Copernicus Land Monitoring studies, including
the monitoring of vegetation, soil and water cover, as well as observation of inland water-
ways and coastal areas [
33
]. Each of the satellites carries a multi-spectral instrument (MSI),
covering 13 spectral bands in the visible, near-infrared (NIR), and short-wave infrared
(SWIR) ranges: four bands at 10 m, six bands at 20 m and three bands at 60 m spatial resolu-
tion [
34
]. The satellite group has a revisit period of 5 days, and the Level-1C product is the
atmospheric apparent reflectance product after ortho correction and geometric precision
correction.
In order to make up for some of the images that could not be used due to excessive
cloud cover and to form a dense time series data stack, we also collected Level-1C products
of the Landsat-8/9 satellite from the United States Geological Survey (USGS) [
35
]. The
payload of the satellite group is the Operational Land Imager (OLI) and the thermal
infrared sensor (TIRS), which can acquire images in 11 bands from visible to short-wave
infrared range, including eight visible and infrared bands with a resolution of 30 m, one
Remote Sens. 2023,15, 3096 5 of 20
panchromatic band with a resolution of 15 m and two thermal infrared bands with a
resolution of 100 m [
36
]. The sensors carried by Landsat-9 have improved radiation
accuracy and have more vital ground object detection ability [
37
]. The satellite group had a
revisit period of 8 days.
In addition to collecting all images that were not obscured or slightly obscured by
clouds during the conflict period (24 February 2022–16 May 2022), we also collected two
images before the conflict and an image after the conflict for comparative analysis. The
specific time and type of optical remote sensing images used in this study are shown in
Figure 2.
Figure 2. Time series optical image of the study area.
The Sentinel-1A satellite launched by ESA was used to map the change in urban
building damage. It is a C-band (
λ=
5.6
cm
) medium-resolution SAR satellite with four
imaging modes: Stripmap (SM), Interferometric Wide Swath (IW), Extra-Wide Swath (EW),
and Wave (WV), with a revisit period of 12 days [
38
]. We acquired Sentinel-1A single look
complex (SLC) images in IW mode from the Alaska Satellite Facility [
39
] from 4 February
to 23 May 2022. These images are all descending orbit images with both VH and VV
polarization, and the spatial resolution is 5 m
×
20 m. The temporal and spatial baselines
between Sentinel-1A image pairs adopted in the study are shown in Figure 3; the positive
and negative of the spatial baseline represent the difference between the baseline direction
and the defined positive direction.
Figure 3. The temporal–spatial baseline of SAR image pairs.
Remote Sens. 2023,15, 3096 6 of 20
The vector data of buildings, land use types and roads used in the experiment came
from the Open Street Map (OSM) platform [
40
]. These data contributed by platform
registrants have the characteristics of inconsistent timeliness and complete alignment
between the vector boundary of buildings and ground objects in satellite images. However,
we tend to pay more attention to macro-scale damage and its change trend in large-scale
armed conflicts. In medium-resolution images, a slight deviation of the building vector
will not significantly impact the evaluation results, so we allow such errors. In addition, we
have collected a large number of live photographs of the damaged buildings in Mariupol
on the Internet platform (these images are from the shared dataset of high-resolution
commercial remote sensing satellites [
41
] and aerial photography from drones, as well
as from the ground, as shown in Figure 4) [
42
,
43
], and determine its spatial location and
damaged degree, and finally, as samples to evaluate the damaged degree of whole urban
buildings.
Figure 4. The type of battlefield images obtained on the Internet.
3. Method
The urban change detection framework in conflict proposed in this paper is shown
in Figure 5, including four parts: optical image preprocessing, SAR image preprocessing,
interest information extraction, and urban damaged change detection and assessment.
First of all, Sen2Cor, SNAP and ENVI were used to preprocess the collected optical
satellite images to form a time-series optical image stack. By observing the spectral re-
sponse value of the urban burning area, the normalized difference fire index (NDFI) was
designed and stacked into the original images as the feature band to reconstruct the spectral
dimension of the original images. Then, the CEM algorithm was used to quickly extract the
burning area in the reconstructed image and accurately identify the urban burning area
with a small amount of manual intervention.
Remote Sens. 2023,15, 3096 7 of 20
Figure 5. Research workflow.
SARscape platform was used to correct Sentinel-1A position with precision orbit
parameter. After calculating the coherence of SAR image pairs, the coherence map was
geocoded by the 1-radian digital elevation model (DEM) of the Shuttle Radar Topography
Mission (SRTM). The Dual polarization normalization coherence index (DPNCI) is designed
as an index to evaluate the damaged degree of urban buildings, and its sensitivity to
damaged buildings is tested by calculating the change rate of DPNCI of burned buildings.
Finally, the spatial location and damage degree of buildings in the photos collected on the
Internet are marked one by one, and the optimal threshold for distinguishing different
damaged degrees of buildings is determined by combining the OSM data so as to form
the analysis and evaluation of the temporal and spatial changes of damaged buildings in
Mariupol.
3.1. Preprocessing
In order to reduce the influence of the atmosphere on the electromagnetic wave,
Sen2Cor was used to conduct atmospheric correction for Sentinel-2A/B Level-1C products
to get Level-2A products, then multi-spectral images with a spatial resolution of 10 m
were obtained by using SNAP to resampling them and format conversion. At the same
time, multispectral images (30 m resolution) and panchromatic images (15 m resolution)
after radiation calibration and atmospheric correction of Landsat-8/9 were fused to obtain
multispectral images with 15 m resolution. Finally, adopt the same boundary vector to
cropped images to obtain the time series optical images.
Used SARscape, the Sentinel-1A image from 4 February 2022, was taken as the master
image and the remaining images as the slave images. The dual polarization coherence
maps between all image pairs were calculated. After geocoding, image cropping, and
Remote Sens. 2023,15, 3096 8 of 20
resampling, the SAR coherence maps (resolution of 5 m) were obtained, which is consistent
with the optical images range.
3.2. Constrained Energy Minimization Algorithm Incorporating the Feature Bands
Am×nmulti-spectral image Swith rbands can be expressed as
S= [s1,s2,s3,· · · ,smn](1)
where siis the spectral vector of the pixel i,si= [si1,si2,si3,· · · ,sir ].
To highlight the information on the burning area, we normalized the difference be-
tween the SWIR1 band and SWIR2 band in Sentinel-2A/B and Landsat-8/9 images and
the NIR band, respectively, to construct two kinds of normalized difference fire indexes
(
NDFIi
) [
44
,
45
]. The reason for choosing the near-infrared band is that there is usually
smoke when the burning event occurs in the conflict zone. Longer wavelength electromag-
netic waves can reduce the influence of scattering to a certain extent.
NDFI1=ρSW IR1−ρNI R
ρSWIR1+ρN I R
(2)
NDFI2=ρSW IR2−ρNI R
ρSWIR2+ρN I R
(3)
where
ρNIR
,
ρSWIR1
and
ρSWIR2
correspond to the bands whose central wavelength is about
0.865 µm, 1.610 µm and 2.200 µm in Sentinel-2A/B and Landsat-8/9 images, respectively.
NDFIi
was inserted into the image as the feature bands
λ1
and
λ2
, and after magnify-
ing them 5000 times, a finite impulse response linear filter
W= [w1
,
w2
,
w3
,
· · ·
,
wr
,
wλ1
,
wλ2]
was designed to minimize the average output energy
WRWT
of the background sample
through the filter by using the known target spectral vector
X= [x1
,
x2
,
x3
,
· · ·
,
xr
,
xλ1
,
xλ2]
as the constraint. In this state, the description of constraints can be followed; that is, while
ensuring a high response to the target spectral vector, excessive attention to background
information can be reduced so as to effectively extract the target. The objective function
and constraint equation is shown in formula (4):
minWRWT
s.t.XWT=1(4)
where R=(1/mn)SSTrepresents the autocorrelation matrix of S.
Under the constraint of
s
.
t
.
XWT=
1, the Lagrange multiplier method is used to solve
the optimal filter coefficient:
WCEM =R−1XT
XR−1XT(5)
where Rrepresents the autocorrelation matrix of S,Xis known target spectral vector.
Ultimately,
WCEM
was applied to each pixel in the image to obtain the distribution of
the target in the image and realize the detection of the target.
In the feature bands images, the pixel value in the burning area is much larger than
that in the non-burning area. Therefore, after the introduction of the feature bands, the
response value of the filter
W
to the burning area is amplified, while the response value of
the non-burning area is suppressed, and the two can be better distinguished.
3.3. Dual Polarization Coherence Change Detection
The damaged buildings can be determined by the actual investigation, image interpre-
tation and other methods. The most accurate method is the actual survey of the disaster
site because closer observation can better characterize the extent of the damage. However,
it is an extremely difficult challenge to assess the extent of damage from a remote sensing
perspective, especially from low- and medium-resolution spaceborne platforms. Therefore,
Remote Sens. 2023,15, 3096 9 of 20
the “damaged change detection” in this study emphasizes explicitly the state that the build-
ing is determined to collapse or not collapse, though it does not accurately detect slight or
moderate damage, such as wall cracks and outer walls being burnt. Nevertheless, from the
point of view of emergency disaster assessment, this may not be important because of the
need to quickly assess the number and spatial distribution of highly damaged buildings. In
the absence of ground survey data and high-resolution optical images, SAR imaging makes
it possible to perform this work because of its all-weather and all-weather characteristics
and the availability of coherence detection changes.
In SAR images, buildings in different states have completely different scattered struc-
tures. Figure 6illustrates common types of damage and the corresponding SAR intensity
images before and after the damage occurred. (a) is a healthy building, usually with strong
backscattering due to the presence of dihedral angles, specular reflection occurs on the
roof, and the walls block the signal to form shadows. (b) is the damaged building, the roof
and wall collapse formed a disorderly distribution so that the SAR echo signal lost the
rule, and the backscattering intensity was significantly weakened. (c) is the damaged type
of some large warehouses or factories. The top of the building is damaged, but the rest
of the structure is relatively intact. The residual walls and ground may form an angular
reflector and cause a strong double-reflection effect [
46
], and the backscattering intensity is
enhanced.
Figure 6.
Comparison of buildings in different damaged states on SAR images before and after
conflict.
Under ideal geometric conditions, the capability of SAR coherence change detection
in building damage detection is better than that of intensity change detection [
47
]. Phase
coherence represents the coherence amount of complex phase signals of SAR images with
two different simultaneous phases at the same position. The coherence coefficient
γ
is 0 to
1, which can be defined as
γ=Ehab∗i
pEhaa∗iEhbb∗i(6)
where
a
and
b
are the relative complex values of the preceding time and the following time
in the interference analysis,
a∗
and
b∗
are complex conjugate values of the images.
E
is the
mathematical expectation.
In the study area, areas with high vegetation coverage, such as farmland and forest
land, often exhibit decoherence due to short-term effects, such as wind disturbance, or
long-term effects, such as vegetation growth. The sea surface, rivers and other water bodies
Remote Sens. 2023,15, 3096 10 of 20
will also cause decoherence because the water surface waves constantly change. In urban
areas, the main features are artificial buildings, and the coherence is usually at a high level
under the condition of not being destroyed by external forces.
The damaged buildings in conflicts can be detected more robustly by synthesizing
different polarization modes. In this study, The DPNCI was constructed by a weighted
combination of the coherence maps in the VV polarization and VH polarization modes.
We selected two SAR images before the conflict, with the image on 4 February 2022, as
the master image and the image on February 16 as the slave image, and calculated the
coherence coefficient within the artificial building land (Red Area in Figure 7, land types
include: commercial, construction, education, gas station, factory, military, religious and
residential). In Figure 7, (a) and (b) are VV and VH polarization coherence maps.
Figure 7. Coherence maps of different polarization before the conflict.
Prior to the conflict, the mean coherence values for VV polarization and VH polariza-
tion were 0.609 and 0.471, respectively. This suggests that the coherence graph calculated
using the VV polarization mode is superior and has a more sensitive index [
48
]. There-
fore, VV should be given more weight than VH when designing the index for damaged
discrimination. The DPNCI is defined as follows:
DPNC I =0.6γVV +0.4γVH (7)
4. Results and Analysis
4.1. Urban Burning Area Detection
To verify the effectiveness of our method, we randomly selected a number of com-
bustion areas and confusing objects (beach and highlighted building tops) on two types of
optical images and compared the response values before and after the method improve-
ment. As shown in Figure 8, IFB-CEM can significantly suppress the response values of
non-burning areas in Landsat-8/9 and Sentinel-2A/B images and amplify the response
values of some pixels in burning areas, which has a more obvious effect in Sentinel-2 images.
(These pixels are usually burning centers, and their response is low in the original CEM
algorithm due to the saturation of their response at bands 11 and 12 of Sentinel-2A/B’s sen-
sor). In the local figure at the bottom right corner of Figure 8, IFB-CEM (split by horizontal
lines) eliminates background interference to a greater extent and preserves target pixels
more completely.
Remote Sens. 2023,15, 3096 11 of 20
Figure 8. The response values of ground objects before and after the method improvement.
We calculated recall, precision and F1 scores under different thresholds by adjusting
response thresholds. Recall is defined as the proportion of correctly extracted burning
areas to the number of all actual burning areas, and precision is defined as the proportion
of correctly extracted burning areas to the number of all extracted burning areas. The F1
score is defined as the harmonic mean of recall and precision. Since the boundary of the
combustion area cannot be accurately divided, we believe that the extracted combustion
area and Ground Truth overlap degree is more significant than 0.8, which means that the
combustion area at this location is successfully extracted. The best response threshold was
determined by the highest F1 score. Table 1shows the best detection effect of CEM and
IFB-CEM methods on each scene. Since the IFB-CEM method suppressed the response of
non-target areas, the error detection rate was significantly reduced, and the precision was
generally significantly improved.
Recall =T P
TP +F N (8)
Precision =TP
TP +FP (9)
F1Score =2·Recall ·Precision
Recall +Precision (10)
where TP,T N and FP are the true positive, true negative and false positive, respectively.
Figure 9shows the temporal and spatial distribution of burning areas during the
conflict. Before and at the beginning of the conflict (before 14 March 2022), the thermal
anomalies were distributed in the Ilyich Iron and Steel Works in the north of the city and
the Azovstal Steel and iron works in the south of the city, indicating that the production
and living activities of the residents were proceeding normally. Images from 14 March
showed that the thermal anomaly in the factory area disappeared, meaning that industrial
production had to stop due to suffering attack and that all the thermal anomalies since
then have been ground objects burning due to the conflict. The Russian forces mainly
attacked from east, west and north three directions. The burning area of the city increased
dramatically in a short time, showing a trend of gradual encirclement and contraction
towards the city center. By the end of March, Russian forces had largely completed the
siege of the Azovstal steel and iron works, after consolidating their positions in the city
center, had launched an offensive against the Ukrainian 36th Marine Brigade stationed at
the Ilyich Iron and Steel Works in the north and had captured the city hall of the Kalmiusky
district. In early April, Russian forces began advancing southwest from the city center
toward the coast in an attempt to encircle and eliminate Ukrainian forces in the port area,
and the burning area shifted accordingly. By the end of April, the thermal anomaly area in
the urban area was reduced to the lowest level, and the remnants of the Ukrainian army all
Remote Sens. 2023,15, 3096 12 of 20
retreated to the Azovstal steel and iron works. The Russian army had taken full control of
the urban area of Mariupol, and the subsequent fighting basically centered on the Azov
steel works, so the burning areas were located in and around the plant. Ukrainian forces
surrendered on 16 May, and Sentinel-2A satellite imagery on 28 May showed no burning
area in or around the Azovstal steel and iron works, bringing the nearly three-month battle
for Mariupol to an end.
Table 1. The optimal extraction results before and after the method improvement.
CEM IFB-CEM
Recall Precision F1 Score Recall Precision F1 Score
4 March 2021 0.959 0.839 0.895 0.980 0.959
0.969(
↑
0.074)
13 January 2022 0.982 0.900 0.939 0.964 0.947
0.955(
↑
0.016)
13 March 2022 0.981 0.912 0.945 0.943 0.963
0.953(
↑
0.008)
14 March 2022 1.000 0.922 0.959 0.979 0.979
0.979(
↑
0.020)
19 March 2022 0.972 0.886 0.927 0.956 0.959
0.957(
↑
0.030)
21 March 2022 1.000 0.877 0.934 0.972 0.986
0.979(
↑
0.045)
22 March 2022 0.965 0.933 0.949 0.948 0.976
0.962(
↑
0.013)
24 March 2022 0.973 0.899 0.934 0.978 0.973
0.975(
↑
0.041)
29 March 2022 0.986 0.907 0.944 0.986 0.986
0.986(
↑
0.042)
3 April 2022 0.984 0.861 0.919 0.984 0.964
0.974(
↑
0.055)
7 April 2022 1.000 0.807 0.893 1.000 0.979
0.989(
↑
0.096)
15 April 2022 0.933 0.667 0.778 0.933 0.933
0.933(
↑
0.115)
30 April 2022 1.000 0.333 0.500 1.000 0.667
0.800(
↑
0.300)
1 May 2022 0.833 0.625 0.714 1.000 0.831
0.908(
↑
0.194)
3 May 2022 1.000 0.667 0.800 0.883 0.883
0.883(
↑
0.083)
8 May 2022 0.833 0.625 0.714 0.700 0.840
0.764(
↑
0.050)
9 May 2022 0.667 0.571 0.615 0.533 0.815
0.645(
↑
0.030)
28 May 2022 0.500 1.000 0.667 0.500 1.000 0.667(--)
Figure 9. The temporal–spatial distribution and area statistics of burning areas during the conflict.
Remote Sens. 2023,15, 3096 13 of 20
4.2. Change Detection of Destroyed Buildings
The calculated DPNCI maps of the time series are shown in Figure 10. Pixel brightness
represents the stability of the building area in Mariupol. With the occurrence and persis-
tence of conflicts, white pixels gradually decrease, indicating that the stability of urban
building areas is disturbed.
Figure 10. Time series DPNCI maps of the study area.
The frequency distribution of the DPNCI in the artificial building land is shown in
Figure 11. Its distribution gradually changes from the right skew (before the conflict) to
the left skew (after the conflict), and the average coherence keeps decreasing. Especially
between 12 March and 24 March, the artificial building showed significant decoherence.
Figure 11. The frequency distribution of the DPNCI in the artificial building land.
Remote Sens. 2023,15, 3096 14 of 20
To verify the sensitivity of the DPNCI to the detection of damaged buildings, we
superimposed the urban burning area extracted in Section 3.1 with the OSM building
vectors, screened out the pixels of buildings located in the burning area, and calculated the
changes of their DPNCI values. It is reasonable to assume that, in modern armed conflicts,
a weapon capable of causing a building to explode and burn would also do great damage
to the structure of the building. Therefore, it is reasonable to consider the buildings in the
burning area as damaged buildings.
In Figure 12, the different colored line segments represent the average DPNCI value
changes of the burned buildings over three different time periods. The blue line showed the
largest decrease in coherence between 12 March and 24 March and remained at a low level,
while the green and black lines also showed the largest decrease in coherence between
24 March and 5 April, 5 April and 17 April, respectively. It is proved that this method can
accurately detect the time node of damaged buildings.
Figure 12. DPNCI value changes of burned buildings.
In order to quantitatively describe the damaged degree and spatial and temporal
distribution of buildings in Mariupol, we graded the damage degree and geographically
located 543 buildings in the collected images and photos. Four damaged levels were
divided according to visual interpretation, as shown in Table 2. 80% of the buildings were
selected to determine the classification threshold, and the remaining 20% were used as
verification data. The confusion matrix of inspection results is shown in Figure 13, among
which the classification accuracy of destroyed buildings is the highest, reaching 69%, and
the overall accuracy is 59.75%.
Remote Sens. 2023,15, 3096 15 of 20
Table 2. Damaged buildings level.
Damaged Degree Description Example DPNCI
Possible damaged or
undamaged
The structure of the building
is intact, and no obvious
cracks or structural changes
can be identified from the
images.
0.60–1.00
Moderate damage
The structure of the building
is relatively intact, with
partially damaged to the top
or sides and no apparent
collapse.
0.45–0.60
Severe damage
The roof and facade of the
building were badly hit, with
extensive collapse, but parts of
the wall structure remained.
0.27–0.45
Destroyed
The building was completely
destroyed in a pile of rubble,
making it difficult to see intact
parts of the walls.
0.00–0.27
Figure 13. Confusion matrix for accuracy verification.
As shown in Figure 14, buildings in Mariupol have been severely damaged in general,
among which 79.2% are severely damaged or destroyed, mainly in Azovstal steel and
iron works and the surrounding central city in the south, Ilyich Iron and Steel Works and
municipal center in the north, and around the port area in the southwest.
Remote Sens. 2023,15, 3096 16 of 20
Figure 14. The change map of building damage in Mariupol.
5. Discussion
Our research shows that the public medium-resolution remote sensing image has the
ability of stable response and large range observation in detecting the change of urban
burning and damaged buildings caused by armed conflict. However, optical satellite
imaging is usually affected by clouds, rain, snow and other weather conditions, especially
in a battlefield environment filled with smoke, which greatly affects the effectiveness of
earth observation. Therefore, the joint observation of optical and SAR images is the key to
solving the problem.
The IFB-CEM we designed allows for better detection of burning areas by more
effectively separating burning and non-burning area responses. However, it is greatly
affected by the preset sample target spectral curve, so it is necessary to choose the pixel
of a burning area with moderate energy as far as possible to balance the detection of the
potential weak burning area and super strong burning center area.
In addition, the DPNCI of medium-resolution SAR images can better depict the
distribution and change trend of damaged buildings in the conflict [
48
]. Combined with
the area and intensity of nighttime lighting in Mariupol [
49
], the damaged buildings are
closely related to nighttime light changes in time and space. Nevertheless, the lack of
high resolution will cause a change of coherence not only from the damage to the building
structure. For example, the coherence of small buildings and buildings with high urban
greening degrees is easily affected by the scattering of electromagnetic waves by other
ground objects, which brings uncertain factors to the detection of damaged grades.
At the same time, we should fully realize the important role of public internet data in
assisting remote sensing image interpretation, which can quickly provide relatively reliable
information before the lack of real statistics. It should be noted that without understanding
the actual situation of the study area, it is extremely difficult and time-consuming to
locate these data from the Internet on the map. In the experiment, we locate the damaged
buildings in the image by looking for iconic ground objects. For this work, perhaps Cross-
Remote Sens. 2023,15, 3096 17 of 20
View Matching technology for Image-Based Ground-to-Aerial Geo-Localization can be
efficiently completed [
50
]. Moreover, the classification of damaged degree also has a
certain degree of subjectivity in this study; the accuracy of damage results obtained by
inversion of the empirical model obtained by visual interpretation and statistics still has
great room for improvement. In fact, the humanitarian disaster survey results released by
UNITAR are a valuable reference, and we also note that these data have been used as an
assessment basis in relevant studies [
20
,
22
]. Figure 15 shows the damaged buildings result
of Mariupol interpreted and released by UNITAR according to the worldview-2 image
on
14 March 2022
[
51
]. Regrettably, we did not adopt it because we do not know what
standard UNITAR uses to divide the damage degree of buildings, and the worldview-2
image and Sentinel-1 image are not acquired simultaneously, and the time difference may
lead to different damage degrees of the same building, especially in the most intense period
of conflict.
Figure 15. Survey of damage to buildings in Mariupol by UNITAR.
According to the above discussion and the inversion results of the damage degree of
the building, we believe that the overall low accuracy is also due to the time difference
between SAR images and acquired images or photos from the Internet.
6. Conclusions
In this study, the city of Mariupol, which was severely affected during the conflict
between Russia and Ukraine, was taken as the research area. Landsat-8/9, Sentinel-1/2,
and open-source community data on the Internet, which were publicly available, were
used to monitor the temporal and spatial changes of urban burning areas and damaged
buildings. The IFB-CEM designed by us has achieved excellent results in detecting burning
areas. Through statistical analysis of the burning area, the intensity of conflict reached
its peak around 22 March 2022. The spatial growth of damaged buildings was positively
correlated with the course of the conflict. By the time the Russian military took full control
of Mariupol (End of May 2022), almost all buildings in the city had been hit, with the
cumulative proportion of destroyed and severely damaged buildings being 79.2%.
Obviously, medium-resolution SAR satellites are not capable of detecting the extent
of damage to small buildings, even though they can provide a large-scale view of the
distribution and trend of damage. The timing mismatch between social media pictures
and remote sensing images is also one of the important reasons that restrict the accuracy of
building damage degree inversion. With today’s convenient access to massive data, it is
Remote Sens. 2023,15, 3096 18 of 20
difficult to locate the location of social media image events only by visual interpretation,
and a more intelligent and automated process is urgently needed to assist this work. In the
future, we will build a more streamlined armed conflict monitoring platform, design more
sophisticated damaged building indicators and rating standards based on higher resolution
SAR images, give full play to the potential of current open source data and public Internet
data in assisting conflict damage detection, expand our detection scope, and carry out
fine-grained conflict damage assessment.
Author Contributions:
Conceptualization, G.J. and Q.H.; methodology, G.J. and Q.H.; software, Q.H.
and Y.X.; validation, G.J., X.X. and H.Y.; formal analysis, H.Y.; investigation, Y.X.; resources, Y.X. and
H.Y.; data curation, Q.H.; writing—original draft preparation, Q.H.; writing—review and editing,
Q.H., X.X. and G.J.; visualization, Q.H.; supervision, X.X.; project administration, G.J. and Q.H.;
funding acquisition, G.J. All authors have read and agreed to the published version of the manuscript.
Funding:
This work was supported by the National Natural Science Foundation of China under
Grants 41474010, 61401509 and 42201492.
Data Availability Statement: Not applicable.
Acknowledgments:
We would like to express our gratitude to the European Space Agency (ESA)
and the United States Geological Survey (USGS), which provided the Sentinel data and Landsat data.
We also thank Open Street Map (OSM) and its users for providing vector data to the study area and
MAXAR for sharing the high-resolution image data set, as well as others, such as Zenger, who shared
the images on Internet platforms. Finally, we would like to express our thanks to the reviewers and
editors who have made valuable comments to improve the quality of this paper.
Conflicts of Interest: The authors declare no conflict of interest.
References
1.
Serhii, A.S.; Vyshnevskyi, V.I.; Olena, P.B. The Use of Remote Sensing Data for Investigation of Environmental Consequences of
Russia-Ukraine War. J. Landsc. Ecol. 2022,15, 36. [CrossRef]
2.
ICRC. Urban Services during Protracted Armed Conflflict: A Call for a Better Approach to Assisting Affected People; International
Committee of the Red Cross: Geneva, Switzerland, 2015.
3.
Lubin, A.; Saleem, A. Remote sensing-based mapping of the destruction to Aleppo during the Syrian Civil War between 2011 and
2017. Appl. Geogr. 2019,108, 30. [CrossRef]
4.
Li, X.; Liu, S.; Jendryke, M.; Li, D.; Wu, C. Night-Time Light Dynamics during the Iraqi Civil War. Remote Sens.
2018
,10, 858.
[CrossRef]
5. Jiang, W.; He, G.; Long, T.; Liu, H. Ongoing Conflict Makes Yemen Dark: From the Perspective of Nighttime Light. Remote Sens.
2017,9, 798. [CrossRef]
6.
Li, X.; Li, D.; Xu, H.; Wu, C. Intercalibration between DMSP/OLS and VIIRS night-time light images to evaluate city light
dynamics of Syria’s major human settlement during Syrian Civil War. Int. J. Remote Sens. 2017,38, 5934. [CrossRef]
7.
Human Rights Watch. Burma: 40 Rohingya Villages Burned Since October. Hum. Rights Watch. [WWW Document]. 2017.
Available online: https://www.hrw.org/news/2017/12/17/burma-40-rohingya-villagesburned-october (accessed on 24 April
2023).
8.
Marx, A.; Windisch, R.; Kim, J.S. Detecting village burnings with high-cadence smallsats: A case-study in the Rakhine State of
Myanmar. Remote Sens. Appl. Soc. Environ. 2019,14, 119. [CrossRef]
9.
United Nations Satellite Centre UNOSAT|UNITAR. Available online: https://www.unitar.org/sustainable-development-goals/
united-nations-satellite-centre-UNOSAT (accessed on 24 April 2023).
10. Bromley, L. Relating violence to MODIS fire detections in Darfur, Sudan. Int. J. Remote Sens. 2010,31, 2277. [CrossRef]
11. Prins, E. Use of low cost Landsat ETM+ to spot burnt villages in Darfur, Sudan. Int. J. Remote Sens. 2008,29, 1207. [CrossRef]
12.
Marx, A.J. Detecting urban destruction in Syria: A Landsat-based approach. Remote Sens. Appl. Soc. Environ.
2016
,4, 30–36.
[CrossRef]
13.
Witmer, F.D.W.; O’Loughlin, J. Detecting the effects of wars in the Caucasus regions of Russia and Georgia using radiometrically
normalized DMSP-OLS nighttime lights imagery. GIScience Remote Sens. 2011,48, 478. [CrossRef]
14.
Li, X.; Li, D. Can night-time light images play a role in evaluating the Syrian Crisis. Int. J. Remote Sens.
2014
,35, 6648. [CrossRef]
15.
Li, X.; Xu, H.; Chen, X.; Li, C. Potential of NPP-VIIRS Nighttime Light Imagery for Modeling the Regional Economy of China.
Remote Sens. 2013,5, 3057–3081. [CrossRef]
16.
Li, L.-L.; Liang, P.; Jiang, S.; Chen, Z.-Q. Multi-Scale Dynamic Analysis of the Russian–Ukrainian Conflict from the Perspective of
Night-Time Lights. Appl. Sci. 2022,12, 12998. [CrossRef]
Remote Sens. 2023,15, 3096 19 of 20
17.
Huang, C.; Hong, S.; Niu, X.; Wu, Q.; Zhong, Y.; Yang, H.; Zhang, H. Mapping of nighttime light trends and refugee population
changes in Ukraine during the Russian–Ukrainian War. Front. Environ. Sci. 2022,11, 33. [CrossRef]
18.
Yelistratova, L.O.; Apostolov, A.; Khodorovskyi, A.Y.; Khyzhniak, A.; Tomchenko, O.V.; Lialko, V.I. Use of Satellite Information
for Evaluation of Socio-Economic Consequences of the War in Ukraine. Ukr. Geogr. J. 2022,2, 11–18. [CrossRef]
19.
Li, J.; Zhou, L.; Ren, C.; Liu, L.; Zhang, D.; Ma, J.; Shi, Y. Spatiotemporal Inversion and Mechanism Analysis of Surface Subsidence
in Shanghai Area Based on Time-Series InSAR. Appl. Sci. 2021,11, 7460. [CrossRef]
20.
Aimaiti, Y.; Sanon, C.; Koch, M.; Baise, L.G.; Moaveni, B. War Related Building Damage Assessment in Kyiv, Ukraine, Using
Sentinel-1 Radar and Sentinel-2 Optical Images. Remote Sens. 2022,14, 6239. [CrossRef]
21.
Washaya, P.; Balz, T.; Mohamadi, B. Coherence Change-Detection with Sentinel-1 for Natural and Anthropogenic Disaster
Monitoring in Urban Areas. Remote Sens. 2018,10, 1026. [CrossRef]
22.
Boloorani, A.D.; Darvishi, M.; Weng, Q.; Liu, X. Post-War Urban Damage Mapping Using InSAR: The Case of Mosul City in Iraq.
ISPRS Int. J. Geo-Inf. 2021,10, 140. [CrossRef]
23.
Corey, S.; Jamon, V. Decentralized, nation-wide, high-frequency war damage mapping using InSAR time series data. In
Proceedings of the AGU Fall Meeting 2022, Chicago, IL, USA, 12–16 December 2022.
24.
Chi, M.; Plaza, A.; Benediktsson, J.A.; Sun, Z.; Shen, J.; Zhu, Y. Big Data for Remote Sensing: Challenges and Opportunities. Proc.
IEEE 2016,104, 2207. [CrossRef]
25.
Campana, S.; Sordini, M.; Berlioz, S.; Vidale, M.; Al-Lyla, R.; Abbo al-Araj, A.; Bianchi, A. Remote sensing and ground survey of
archaeological damage and destruction at Nineveh during the ISIS occupation. Antiquity 2022,96, 436. [CrossRef]
26.
Ali, S.; Levin, N.; Crandall, D. Utilizing remote sensing and big data to quantify conflict intensity: The Arab Spring as a case
study. Appl. Geogr. 2018,94, 1–17. [CrossRef]
27. Mariupol. Available online: https://en.wikipedia.org/wiki/Mariupol (accessed on 24 April 2023).
28.
CNN. CNN (Ukraine-Satellite-Images). 2022. Available online: https://edition.cnn.com/interactive/2022/03/world/ukraine-
satellite-images/ (accessed on 24 April 2023).
29.
CNN. What Does Putin Want in Ukraine? The Conflict Explained. 2022. Available online: https://edition.cnn.com/2022/02/24
/europe/ukraine-russia-conflict-explainer-2-cmd-intl/index.html (accessed on 24 April 2023).
30.
TASS (Military Operation in Ukraine). Putin Declares Beginning of Military Operation in Ukraine. 2022. Available online:
https://tass.com/politics/1409329 (accessed on 24 April 2023).
31.
Neta, C. Crawford Reliable Death Tolls from the Ukraine War Are Hard to Come by—The Result of Undercounts and Manipulation.
The Conversation, 4 April 2022.
32. Copernicus Open Access Hub. Available online: https://scihub.copernicus.eu (accessed on 24 April 2023).
33.
USGS. EROS Archive—Sentinel-2|U.S Geological Survey. Available online: https://www.usgs.gov/centers/eros/science/
usgseros-archive-sentinel- 2?qt-science_center_objects=0#qt- science_center_objects (accessed on 13 February 2023).
34.
European Space Agency Sentinel-2 User Handbook. 2015. Available online: https://sentinels.copernicus.eu/web/sentinel/
userguides/document-library/-/asset_publisher/xlslt4309D5h/content/sentinel-2-user-handbook (accessed on 13 February
2023).
35. USGS EarthExplorer. Available online: http://earthexplorer.usgs.gov (accessed on 24 April 2023).
36.
Landsat 8 Data Users Handbook|U.S. Geological Survey. Available online: https://www.usgs.gov/landsat-missions/landsat-8-
data-users-handbook (accessed on 13 February 2023).
37.
Maseka, J.G.; Wulder, M.A.; Markham, B.; McCorkel, J.; Crawford, C.J.; Storey, J.; Jenstrom, D.T. Landsat 9: Empowering open
science and applications through continuity. Remote Sens. Environ. 2020,248, 111968. [CrossRef]
38.
User Guides—Sentinel-1 SAR—Sentinel Online—Sentinel Online. Available online: https://sentinel.esa.int/web/sentinel/user-
guides/sentinel-1-sar (accessed on 13 February 2023).
39. Alaska Satellite Facility. Available online: https://search.asf.alaska.edu (accessed on 24 April 2023).
40. Open Street Map. Available online: www.openstreetmap.org (accessed on 13 February 2023).
41.
Satellite Images Map of Mariupol (Mapping.jp). Available online: https://ukraine.mapping.jp/mariupol.html (accessed on 11
April 2023).
42.
Twitter Search Image “Mariupol”. Available online: https://twitter.com/search?q=mariupol&src=typed_query&f=image
(accessed on 23 July 2022).
43. Facebook Search “Mariupol”. Available online: https://m.facebook.com/profile.php?id=108052062548731 (accessed on 23 July
2022).
44.
Garcia, M.J.L.; Caselles, V. Mapping Burns and Natural Reforestation Using Thematic Mapper Data. Geocarto Int.
1991
,6, 31.
[CrossRef]
45.
Key, C.; Benson, N. Landscape Assessment: Ground Measure of Severity, the Composite Burn Index; and Remote Sensing of
Severity, the Normalized Burn Ratio. In FIREMON: Fire Effects Monitoring and Inventory System; USDA Forest Service, Rocky
Mountain Research Station: Fort Collins, CO, USA, 2005; Volume 2004.
46.
Matsuoka, M.; Yamazaki, F. Use of Satellite SAR Intensity Imagery for Detecting Building Areas Damaged Due to Earthquakes.
Earthq. Spectra 2004,20, 975–994. [CrossRef]
Remote Sens. 2023,15, 3096 20 of 20
47.
Takeuchi, S.; Suga, Y.; Yonezawa, C.; Chen, A.J. Detection of Urban Disaster Using InSAR: A Case Study for the 1999 Great Taiwan
Earthquake. In Proceedings of the IEEE 2000 International Geoscience and Remote Sensing Symposium, Honolulu, HI, USA,
24–28 July 2000.
48.
Karimzadeh, S.; Mastuoka, M. Building Damage Assessment Using Multisensor Dual-Polarized Synthetic Aperture Radar Data
for the 2016 M 6.2 Amatrice Earthquake, Italy. Remote Sens. 2017,9, 330. [CrossRef]
49.
Yelistratova, L.A.; Apostolov, A.; Movchan, D.M. Using night illumination images based on remote sensing data for the socio-
economic assessment of a besieged city (Mariupol City, (Ukraine) as an example). In Natural Resource Potential, Ecology, and
Sustainable Development of Administrative Units of the Republic of Latvia and Ukraine Amidst EU Legislative Requirements; Baltija
Publishing: Riga, Latvia, 2022; p. 82. [CrossRef]
50.
Hu, S.; Feng, M.; Nguyen, R.M.H.; Lee, G.H. CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial
Geo-Localization. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt
Lake City, UT, USA, 18–22 June 2018; pp. 7258–7267. [CrossRef]
51.
Damage Assessment Overview Map—Livoberezhnyi and Zhovtnevyi Districts, Mariupol City, Ukraine. Available online:
http://unosat.org/products/3371 (accessed on 24 April 2023).
Disclaimer/Publisher’s Note:
The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
Available via license: CC BY
Content may be subject to copyright.
