Content uploaded by Colin Chambers
Author content
All content in this area was uploaded by Colin Chambers on Dec 14, 2023
Content may be subject to copyright.
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Development of a novel methodology to assess and communicate the effectiveness of risk control
measures and strategies applied to a Covid-19 scenario
Authors: Colin Chambers, April Lockhart, Zoe Chaplin
Corresponding author: Colin Chambers – colin.chambers@hse.gov.uk
Institution: Health and Safety Executive Science and Research Centre, Harpur Hill, Buxton, Derbyshire, SK17 9JN.
This paper describes the development of a holistic, hierarchical risk assessment and risk communication
methodology, based on a novel variation of the bowtie diagram, which the authors have named the ‘weighted
bowtie diagram’, initially devised as part of HSE’s response to the recent Covid-19 pandemic. This paper presents
a concept demonstration of the proposed methodology using an example Covid-19 scenario.
The objectives in developing the enhanced bowtie diagram methodology were to: facilitate a holistic risk
assessment, covering a wide range of factors associated with both the prevention and mitigation of the top event
being evaluated; to explore and compare the effectiveness of different risk control measures and risk control
strategies (barriers and threat/consequence pathways); and to communicate risk assessment outcomes to
audiences with different levels of understanding of the issues involved.
The proposed methodology is based on a novel augmentation of the bowtie diagram, that results in a ‘weighted
bowtie diagram’, augmented by means of a visual representation of risk/risk reduction and the use of proven risk
assessment techniques such as Fault Tree Analysis (FTA) and Event Tree Analysis (ETA).
There are three novel aspects of this proposed methodology:
1. The detailed analysis of individual barriers to estimate their effectiveness (weighting) and their
strengths and weaknesses. The detailed barrier analysis can result in advice on how the barrier
effectiveness could be improved by suggesting recommended good practice or by highlighting poor
practice that could result in a reduced barrier effectiveness.
2. The use of the weighted bowtie diagram to communicate risk assessment outcomes. For example,
presenting visually the effectiveness of individual barriers and/or of threat /consequence pathways
(strategies) to inform which are likely to be the most effective at reducing risk.
3. How the weighted bowtie diagram is designed to illustrate weighting of risk using visualisation
techniques such as colour and line thickness on the threat and consequence pathways. This weighting
can be used to depict relative risk reduction implemented by each barrier and threat consequence
pathway presented in the ‘weighted bow-tie diagram’.
The authors applied the proposed methodology to a fictitious Covid-19 scenario, namely a person attending the
gym to demonstrate how the proposed methodology can be applied only and not as a risk assessment of a ‘real
scenario’. The reason for this choice of scenario was to demonstrate the flexibility of the methodology in being
applicable to both to public health and workplace scenarios. A qualitative approach based on the authors’
judgement combined with information from a literature review was used for this example concept demonstration.
However, where suitable data is available the weighted bowtie diagram methodology is expected to support
numerical analysis. The ability to use numerical data supports application of the proposed methodology in the
process sector where use of traditional bowtie diagram analyses is well established.
The authors aim to identify opportunities for further trials of the proposed methodology, to explore its robustness
and versatility.
Key words: Covid-19, holistic risk assessment, risk communication, risk assessment techniques, weighted bow-
tie diagram, risk control strategies
Introduction
Many techniques can be used to assess risks associated with the Covid-19 pandemic, but no overarching risk assessment
methodology used for this purpose has been identified to date by the authors.
As part of the Covid-19 pandemic response this work was commissioned with the following aims:
• Develop a risk assessment and risk communication methodology that can be applied to a Covid-19 scenario using
either qualitative and/or quantitative techniques.
• Determine the effectiveness of different risk control strategies in reducing risk; and
• Determine how individual barriers could fail and how the extent and/or likelihood of that failure could be reduced.
Risk assessment techniques such as fault tree analysis (FTA), event tree analysis (ETA), failure mode and effects analysis
(FMEA), and layers of protection analysis (LOPA), as described in BS EN IEC 31010:2019, can be used to evaluate different
aspects of the Covid-19 pandemic risk. Techniques such as FTA and LOPA can be used to examine the top events that lead to
a single stated consequence. Other techniques, such as ETA, allow the evaluation of multiple consequences given that the
same top event has occurred.
One risk assessment technique, the bow tie diagram (BTD), allows the depiction of a top event complete with its threats,
consequences, relevant barriers, and their associated degradation factors and controls (failure modes and recovery
mechanisms) in one diagram, CCPS 2018. BTDs are often used to depict a top event risk landscape and communicate
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
associated risk control strategies to interested parties. Hence, BTD has the potential to form the basis of a holistic risk
assessment methodology suitable for application to a wide range of hazards in a wide range of industries, CCPS 2018. Two
limitations associated with the traditional BTD technique are that 1) visual clarity can easily be reduced when presenting
multiple top event threats, barriers, and associated escalation events, CCPS 2018, and 2) the traditional BTD does not indicate
any visual or numeric measure of risk or risk reduction associated with the top event and/or risk reduction implemented by the
barriers. However, in recent years, the BTD technique has been augmented to perform numerical analysis of risk, as seen in
Emery 2014 and CGE 2022.
This work presents what the authors believe is an intuitive way to address the issues highlighted here and communicate risk
assessment outcomes to interested parties using a novel variation of the BTD developed for this work, namely the weighted
bowtie diagram (WBTD) and associated techniques. The WBTD acts as a risk communication technique that visually presents
the effect of barriers on the likelihood of the top event and/or its outcome(s) occurring. The WBTD methodology also includes
the use of other established risk assessment techniques to determine the barrier effectiveness, and assess factors associated
with the barrier that could impede its effectiveness at reducing risk as part of holistic risk analysis. This study presents a
concept demonstration used to assess and communicate the risks associated with a Covid-19 scenario.
In this section, the WBTD methodology process, summarized in Figure 1, is presented via an example scenario ‘attending the
gym’. This example scenario looks beyond the duties of an employer under the Health and Safety at Work Act (1974) in terms
of the types of suggested barriers that an employer might reasonably be expected to consider implementing to prevent the
spread of infection in the workplace. Some of the barriers included may be associated with the wider scope of public health
risks outside of the workplace to demonstrate the potential versatility of the weighted bow-tie diagram methodology in
assessing and communicating complex, multifaceted, real work risks in a structured and accessible manner. The scenario
includes both examples of the use of supporting risk assessment techniques and the assessment parameters used as part of the
proposed weighted bow-tie diagram methodology. This explanation includes the definition of the terminology used, which is
based in part on the traditional BTD method, CCPS 2018 and in part on the LOPA method CCPS 2001 where relevant.
Figure 1: WBTD diagram methodology process flowchart
Identify the top event and consequences
A BTD models a ‘threat – top event – consequence’ approach with barriers, degradation factors and their controls, controlling
the risks represented in the BTD. The WBTD methodology applies the same approach but with added enhancements to visually
represent the relative risk and risk reduction implemented by these barriers.
The first stage in the process is to identify the hazard and top event and their threats and consequences CCPS 2018. This study
defines the term scenario as the combination of the threat(s), top event, and its consequence(s) that are to be assessed.
Using techniques described later in this paper, several threats were identified that could lead to the identified top event, which
in the authors’ opinion was representative of everyday tasks performed by members of the public ‘attending the gym’. The
activity of attending the gym was chosen by opportunity sampling, as described by Saunders et al. 2012 as a common sampling
method based on the options identified by the authors at the time, and therefore, are not representative of all possible activities
that might lead to contracting SARS-CoV-2.
The consequence(s) could be either predetermined or identified using techniques such as brainstorming and FMEA. In the
example scenario, the brainstorm technique was used to identify a range of potential consequences, and options were limited
to simplify the assessment.
The Covid-19 scenario to be assessed here is:
An individual is infected with SARS-CoV-2 while attending the gym, and that subsequently the individual is either:
• Admitted to hospital with Covid-19 and recovers, or
• Admitted to hospital with Covid-19 and suffers fatality resulting from Covid-19 as primary or secondary cause.
Other consequences are possible, but the following assessment is limited to the consequences stated prior for simplicity.
Identify the top event threats
It is important to state assumptions made in a risk assessment to clarify and document the basis of the assessment and
subsequent decision making. A list of assumptions was made to define the scope of the ‘attending the gym’ scenario, e.g. does
not include travel to/from the gym.
Identify the
top event and
consequences
Identify the
top event
threats
Identify barriers,
RGPs and
degradation
factors and their
controls
Produce
traditional and
simplified bow
tie diagrams
Estimate of
risk and
uncertainty
Create a
weighted bow
tie diagram
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
One of many established techniques can be used to identity the top event threats; however, it should be noted that consideration
of the consequences can also help identify relevant threats. The demand tree technique was used to identify the top event
threats for this example scenario, see Figure Error! Reference source not found.2. The demand tree is a systematic technique
that can be used to identify top event threats and is analogous to a fault tree comprising solely of logical ‘OR’ gates, see HSE
2009.
Note that in this work, we have implemented rules normally associated with LOPA, CCPS 2001, to help ensure consistency
in how risk and, hence, risk reduction are dealt with in the WBTD. One notable difference is that the WBTD can account for
multiple outcomes; how this is accounted for is discussed later in this paper. For example, in this case, a threat is only assessed
in the WBTD if it results in the top event occurring if left unchecked.
The selected threats carried forward to the assessment are highlighted in Figure 2: contact with surfaces at external or internal
doors and turnstile, using gym apparatus, use of locker, use of shower, and presence in a poorly ventilated indoor environment
for 90 minutes (the estimated time an individual spends at the gym and used for demonstration purposes only). To simplify
the WBTD assessment, the threats taken from the demand tree have been grouped together where possible, and others have
been screened out for other reasons, such as their being considered unlikely to occur when compared with other threats, or that
the individual does not have close contact with staff or other gym users during their session during the pandemic.
Figure 2: example demand tree used for the systematic identification of threats associated with ‘attending the gym’ scenario
Identify barriers, recommended good practice and degradation factors and their controls
Barriers can be either preventive or mitigative. Preventive barriers appear on the left-hand side of the WBTD and mitigative
barriers appear on the right-hand side of the WBTD. Barriers can be identified using several techniques including brainstorms,
checklists, and FMEA.
For the ‘attending the gym’ scenario, preventive and mitigative barriers were identified based on the top event threats and
consequences with the aid of a 3-stage tabular approach devised for this purpose, see Table 1 and Table 2. This approach starts
with either the threat or consequence in the first column, and then, the logical mitigation or prevention action for that threat or
consequence is stated, tersely, in the second column, which is used as a prompt to identify the physical or procedural barrier
in the third column.
Table 1: 3-stage example brainstorm approach used to identify preventive barriers
Top event threat
Threat prevention
Potential barrier
Contact with surfaces at external or
internal doors and turnstile
Prevent surface contact
Mitigate surface contact
Use non-contact entry
Wash/sanitise hands after entry
Using gym apparatus
Prevent contact with equipment
Mitigate contact with equipment
Cannot prevent contact so clean
equipment before and after use
Sanitise hands between use
Use of locker
Prevent direct contact with locker
Mitigate contact with locker
Eliminating use of lockers is unlikely
so increase frequency of cleaning
Sanitise hands between use
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Use of showers
Prevent direct contact
Mitigate contact
Regular cleaning of shower cubicle
Limit shower occupancy duration
Clean shower frequently after use
Sanitise hands between use
Presence in a poorly ventilated, indoor
environment for 90 minutes
Prevent build-up of the virus carried
by aerosol/small droplets leading to
potential infection
Limit duration of each visit
Limit access to worst areas in gym
Monitor CO2 in different areas of the
gym
Passive ventilation, e.g., open
windows
Table 2: 3-example stage brainstorm approach used to identify mitigative barriers.
Top event consequence
Consequence mitigation
Potential barriers
Admitted to hospital with Covid-19
Improve immune response such that it
is likely the infection is asymptomatic
or mild
Ensure vaccinated as soon as possible
Take steps to support health and
wellbeing at home when symptomatic
in effort to reduce likelihood of
hospital admission
Provide tips for self-care at home,
including monitoring of symptoms
Fatality from Covid-19 as primary or
secondary threat
Improve immune response such that it
is likely the infection is asymptomatic
or mild
Ensure vaccinated as soon as possible
Take steps to support health and
wellbeing at home when symptomatic
to reduce the likelihood of hospital
admission
Provide tips for self-care at home,
incl. monitoring of symptoms with
admission to hospital if condition
deteriorates (early intervention
decreases mortality)
When in hospital, all relevant
measures taken to support a positive
outcome
Hospital treatments and care
including, but not limited to, drug
therapy and mechanical ventilation as
required
Using the outcomes determined by using both the three-stage tabular approach, and with reference to government published
guidance and information, see GOV.UK 2021, the following barriers were used in the ‘attending the gym’ scenario:
Table 3: Barriers implemented in the ‘attend the gym’ example scenario
Preventive
Mitigative
Barriers
Surface cleaning
Wash/sanitise hands
Occupancy control
Physical distancing
Vaccination
Hospital treatment
The above does not list face coverings as a preventive barrier because according to government publications on controls in
this type of environment, it is not recommended for people undertaking exercise GOV.UK, (2022).
Due to uncertainty of their effectiveness or concerns as to whether a potential barrier would be applied effectively in the
scenario being assessed, no credit can be reasonably claimed for some identified barriers. As such, these barriers are referred
to as recommended good practice (RGP) in the WBTD methodology.
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Table 4: RGP factors used in the ‘attending the gym’ scenario
Preventative
Mitigative
Recommended
Good Practice
(RGPs)
Passive ventilation – no credit claimed due to
the assumed likely low effectiveness and high
uncertainty of this barrier for the hypothetical
gym in the example scenario. However, this
remains an item in the diagram as a reasonable
step to take.
Treatment at home – no credit claimed due to a
lack of evidence available supporting its
effectiveness but is included because there may
be benefit in communicating self-care
principles to the individual when they receive
confirmation of a positive Covid-19 test result.
A degradation factor can be thought of as a barrier failure mode, and degradation controls can be thought of as the failure
mode recovery mechanisms. The barrier efficacy is defined here as the theoretical likelihood that it will implement the
identified amount of risk reduction without accounting for the negative impact from any degradation factors. The barrier
effectiveness is defined here as the likelihood that it will implement the identified amount risk reduction having accounted for
the potential negative impact from any degradation factors in combination with the degradation controls where relevant.
Each barrier can have multiple degradation factors. However, ideally their total number is rationalised by means of screening
and/or combining them to include only useful/dominant degradation factors and controls in the risk assessment.
Degradation factors and their controls can be systematically identified using established risk assessment techniques such as:
FTA and FMEA. FTA was used in the ‘attending the gym’ scenario. Figure 3 presents a fault tree used to assess the ‘occupancy
control’ barrier and subsequently determine its degradation factors and their controls.
Figure 3: example use of FTA to identify barrier degradation factors and their controls for the ‘attending the gym’ scenario
In this case, two degradation factors were assigned based on the results of the FTA. Degradation factor 1 is: ‘occupancy control
measure inadequately implemented.’ Degradation factor 1 controls are: ‘the gym provides adequate staff training to implement
the degradation factor controls’ and ‘the gym adequately resources and supports staff to implement the occupancy controls.’
Degradation factor 2 is the ‘failure of gym clients to adhere to occupancy controls.’ Degradation factor 2 controls are: ‘amend
and communicate to gym members the gym terms and conditions’ and ‘proactively communicate occupancy controls to clients
verbally, in writing, and by use of prominent signage.’.
Risk assessment is often an iterative process, Ellerman et al., 2022. Later analysis might find that some of the initially selected
barriers may not be as effective as required or when first thought of. Therefore, although this scenario is presented linearly, it
is unlikely to reflect how it will be applied in each case.
Risk Modification Factors
It is proposed that risk modification factors (RMF) are used in the WBTD methodology to account for inherent factors not
implemented to control the risks. RMFs can be thought of as analogous to LOPA conditional modifiers, CCPS 2001. RMFs
need to be more flexible than their LOPA counterparts because they need to account for a wide range of factors, some of which
will have a negative impact on the risk assessment outcome, and some will have a positive impact.
Occupancy
control fails
Failure of clients to
adhere to
occupancy controls
Unaware of
occupancy contr ols
Not required to
according to T&Cs
Base 3 Base 4
Occupancy controls
implemented
inadequately
Base 1 Base 2
Failure to
adequately resource
and support st aff
Failure to provide
adequate training to
staff
Chooses not to
adhere to control
Base 5
Degradation
factors
Basis of degradation
factor controls
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
An RMF can be applied to an individual or can be common to groups of people. Example RMFs in relation to Covid-19 could
include pre-existing health conditions, such as chronic obstructive pulmonary disease (COPD), blood cancer, age, sex, or to
account for different outcomes being inherently more or less likely to occur than each other. Without accounting for RMFs,
the assessment of Covid-19 scenarios may not be representative of the range of at-risk people.
For Covid-19 scenarios, where supporting data appear limited or highly uncertain, it is suggested that the RMF could depict a
simple positive or negative impact on the risk outcome in a qualitative sense. It is likely that the contribution of RMFs to risk
is based on expert opinion. RMFs can be preventive or mitigative.
Produce traditional and simplified BTDs
The next stage in the WBTD methodology process is to collect the information gathered thus far and populate a BTD. It is
proposed that two BTDs be produced in the first instance. First, a traditional BTD that contains all the risk assessment
parameters, namely barriers, degradation factors and their controls, RMFs, and RGPs. The traditional BTD is the backbone of
risk assessment, and once the assessment has been performed, it will likely be updated when new information becomes
available. Second, a simplified BTD will be produced with much of the detailed information abstracted. The simplified BTD
forms the basis for the WBTD. Numerous WBTDs may be generated from a particular detailed traditional BTD, depending
on which aspect of the assessment is to be communicated at that time.
Traditional BTD
A traditional BTD is a graphical threat-consequence analysis method used to assess risk pathways from top event threats to
their consequences and visually communicate the risk landscape (see Figure 4).
Threat (1)
Threat (2)
Threat (3)
Sources of risk
Consequence
(1)
Consequence
(2)
Consequence
(3)
Areas of impact
Common barrier
Prevention Barriers
Degradation
Factor Degradation
Factor
Mitigation Barriers
Degradation Controls
Threat pathways Consequence pathways
Top Event
Hazard
Figure 4: Typical bow tie diagram concepts
The top event is at the centre of the BTD. The top event threats are to the left of the diagram and the top event consequences
are to the right of the diagram. Prevention barriers reduce the likelihood of a threat leading to the occurrence of a top event,
thereby reducing the likelihood of the consequence occurring. Mitigative barriers reduce the likelihood of top events and
potentially reduce their severity. Degradation factors and their controls represent how a barrier might fail, and consequently,
how that failure could be prevented/mitigated. However, there are three limitations to the traditional BTD that this study aims
to address.
1. The power of BTD analysis is the depiction of the overall top event scenario, with the assessment notionally flowing
on assessment pathways from threats to top event to consequences. This notional flow could direct the assessors’
thought process to making judgements about the number of barriers in place rather that adequacy of the presented
barriers in the context of the top event and its consequences occurring.
2. The visual clarity of the BTD is diminished when presenting multiple top event threats, barriers and associated
degradation events (and other associated information often presented on the BTD); and
3. A traditional BTD does not visually or numerically indicate a measure of risk and/or risk reduction implemented by
the barriers considered in the BTD. This can make the comparison of different risk reduction strategies problematic.
Weighted BTD
The authors propose that the three limitations of the traditional BTD noted previously can be addressed using the WBTD
methodology. The WBTD process involves presenting only the minimum information in the WBTD to communicate the
message the risk assessor wishes to get across to the target audience. The WBTD presents the relative risk visually, both before
and after each barrier, as a method conveying not only the overall risk of each threat leading to the top event occurring, but
also to show the contribution of each barrier to that overall risk reduction. With the effectiveness of each barrier being presented
visually, this can help the risk assessor consider the ‘quality’ of each barrier not just the number of them associated with any
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
threat or consequence. Whether the analysis is semi-quantitative or qualitative, the magnitude of risk can be depicted at any
point throughout the WBTD See figure 9.
The estimated magnitude of the risk associated with each threat can be used to set the initial weighting for each pathway. When
a qualitative assessment is performed, a scale, key, or legend can be provided. The options initially considered to depict the
weighting were visually varied line thicknesses, varying colours, or both.
The WTBD can result in what the authors believe is relatively easy to explain and a relatively uncluttered diagram and is
considered by the authors to be an improvement in visual clarity over the traditional BTD, thus aiding risk communication.
‘Attending the gym’ scenario diagrams
Using the approaches described previously, for the ‘attending the gym’ scenario to derive the barriers, etc., Figure 5 shows the
traditional BTD, and Figure 6 shows the simplified BTD (precursor to the WBTD).
Figure 5: Traditional detailed BTD for the ‘attending the gym’ scenario assessment
The detailed BTD for the ‘attending the gym’ shown in Figure 5 demonstrates the difficulty of picking up all the details from
a BTD without zooming in or printing to large paper sizes.
In line with the proposed methodology, a simplified diagram was created from the traditional BTD, as shown in Figure 6.
Use of showers
Presence in poorly
ventilated, indoor
environment
Using gym apparatus
Contact with surfaces
at external/internal
doors and turnstile
Use of locker Individual is infected
with SARS-CoV-2
SARS-CoV-2 in
the human
population
Open doors,
vents and
windows
Occupancy
control c
Physical
distancing
(2 m) d
Cleaning
hands b
Surface
cleaning a
Attending the gym
Individual is admitted
to hospital then dies
from COVID-19 as
primary or secondary
cause
Individual is admitted
to hospital with
COVID-19 and
recovers
Blood
Cancer
Fatality Hospital
treatment
2
Vaccination
1
At home
care
Figure 6: Simplified BTD for the ‘attending the gym’ scenario assessment.
It is recommended the simplified BTD is used to apply weighting following the determination of risk (or relative risk, as
appropriate).
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Estimation of risk and uncertainty
A literature search was performed to identify data, for the effectiveness of barriers and the frequencies that each threat would
likely occur, that could be fed into the WBTD for the Covid-19 example scenario. Uncertainties associated with the data were
also considered. The authors note that it is important to acknowledge uncertainty in determination of risk or relative risk.
The outcome of this literature search was that there was insufficient data to support a quantitative risk assessment, and what
data was available was wide-ranging across different sources suggesting that this data was at best indicative rather than
definitive. Due to the apparent lack of suitable data, the authors chose to take a qualitative approach for the example scenario
‘attending the gym’ risk assessment. Table 5 and figure 9, present the 7-point qualitative scale used for relative risk in the
‘attend the gym example’ for demonstration purposes only.
The frequency of occurrence of each threat would ideally be determined using supporting data from relevant literature or other
credible sources such as expert judgement, although expert judgment can vary widely between different experts, see Freeman
2021. For the example ‘attending the gym’ scenario, the initial threat frequencies were selected from a 7-point qualitative
scale, see Table 5 and Figure 9, by the research team members using their opinion.
For this risk assessment exercise only, it was assumed that the effectiveness of each barrier would result in a reduction of risk
of one or more scale points, e.g. from ‘low’ to ‘very low’ or ‘moderate’ to ‘very low’, this is for demonstration purposes only.
This approach can be refined by consideration of criteria often used in the LOPA method for determining whether any risk
reduction can be claimed for a barrier and what the level of risk reduction would be, CCPS 2001.
In a WBTD, risk is represented by each threat or consequence pathway line thickness or weight. This initial or unmitigated
risk, or frequency, is the thickness of the line from the threat and the starting risk for the first barrier. This initial risk is modified
by the barrier effectiveness, which is represented by the barrier probability of failure on demand (PFD) for a demand-based
barrier. The barrier PFD is referred to as the barrier weighting factor in this paper, given that it directly determines the line
weight or thickness immediately after the barrier. The amount of reduction in the threat pathway line thickness is proportional
to the barrier weighting factor but is not necessarily directly equivalent. If a barrier reduces the risk by a weighting factor of
0.5 (50%), then the line thickness could be reduced by 50% to represent this level of risk reduction. However, visually that
could be problematic because after several barriers are evaluated the resultant changes in line thickness might not be perceptible
when presented to a risk assessment target audience. The authors suggest that one way to address this problem is to define
multiple line thicknesses that can be distinguished from each other in a power-point presentation and assign a range of
probabilities covering the range used in the risk assessment. The 7-point qualitative scale, see Table 5 and Figure 9, used in
the ‘attending the gym’ example scenario was chosen because the research team could easily differentiate between each line
thickness visually.
The total frequency of the top event occurring is the sum of the individual threat frequencies leading to the top event occurring.
This was taken as the starting frequency for each top event consequence evaluated in the WBTD, see Figure 7.
Top Event
Consequence
(1)
Consequence
(4)
Consequence
(2)
Consequence
(3)
Threat (1)
Threat (4)
Threat (2)
Threat (3)
Hazard
Combined likelihood
of hazardous event
occurring
Likelihood of each
outcome occuring given
the likelihood of the
hazardous event
occurring
Example Risk Assessment Approach
Figure 7: bow tie diagram risk calculation approach, which is analogous to that used for LOPA
Different top event consequences can have a different inherent likelihood of occurring given the same top event frequency.
For example, in a study conducted between March 2020 and March 2021, it was found that 25% of patients hospitalized with
Covid-19 in England died in the hospital Williams et al. 2021. This implies that the likelihood of consequences other than
fatality at that point in time is approximately 75%. The WBTD methodology accounts for this using an RMF, which is in this
case analogous to the CM ‘probability of fatality’ often used in LOPA.
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Table 5: possible barrier and RMF effectiveness scale
Effectiveness
qualitative
descriptor
Extremely
high
Very high
High
Moderate
Low
Very low
Extremel
y low
Effectiveness
range
100-86%
86-71%
71-57%
57-43%
43-28%
28-14%
14-0%
Average
94%
78.5%
64%
50%
35.5%
21%
7%
For example, the effectiveness of the barrier ‘occupancy control’ was assumed to be ‘high’ (64% average effectiveness, see
Table 5) at reducing the likelihood of becoming infected with SARS-CoV-2 while attending the gym; for the purpose of this
assessment only.
Furthermore, the likelihood of a top event or consequence occurring without any barriers or RMFs is needed to support the
estimation of risk. There is information in the public domain about virus prevalence in the population that varies over time,
but not to a level which supported quantitative analysis for the example scenario ‘attending the gym’ at a specific location and
date. This pointed towards a qualitative assessment of Covid-19 and this ‘attending the gym’ scenario.
Some measures are, by their nature, likely to be considered less effective in reducing the likelihood of the top event occurring
than others. In the context of our ‘attending the gym’ scenario, an example is hand washing. As the pandemic progressed, there
was an increasing consensus that transmission of the virus was potentially dominated by exposure to airborne particles, and
less so by transference of the virus from the contaminated surface to the hand to face, CDC 2021. Although there is likely to
be some reduction in risk with mask wearing, it might not achieve the level of risk reduction from measures such as hand
washing and physical distancing. This is illustrated below assuming a very high effectiveness of hand washing and a relatively
low effectiveness of masks (such as those typically worn by the public, without a fit test).
Table 6: example indicative data used to support selection of barrier (values used are for demonstration of the methodology
only due to significant variability depending on the source referenced)
Airborne transmission
Surface transmission
Likelihood of obtaining an infection without barriers
10%
0.1%
barrier
Face covering
Hand washing
Effectiveness of barrier
20%
99%
Likelihood of obtaining an infection with barrier
10% × (1 - 20%) = 8%
0.1% × (1 - 99%) = 0.001%
Absolute risk reduction
10% - 8% = 2%
0.1% - 0.001% = 0.099%
The example in Table 5 demonstrates the requirement to consider both barrier effectiveness and the likelihood of the top event
occurring for that threat, because otherwise the risk assessment may not provide sensible results. Absolute numbers of
likelihood of obtaining an infection without barrier effectiveness estimates, as used in the above, are for demonstration
purposes only.
From the data obtained, the effectiveness of individual barriers can vary widely, especially those based on expert judgment,
and is dependent on many factors, such as the way they are implemented Freeman, 2021. In the WBTD methodology, such
issues can be considered in the degradation factor component of the detailed barrier analysis discussed in this paper.
Risk assessment is often an iterative process based on the authors experience, and with weighting, even if only qualitative, it
might help identify where to apply improvements to, or where further controls are required. This can facilitate what-if analyses
to compare different control strategies.
Other analysis options available at this stage include revisiting the selection of barrier combinations in any threat pathway or
consequence pathway (threat control/mitigation strategies) using risk assessment techniques such as ETA, as illustrated in
Figure 8.
For example, Figure 8 presents an ETA that considers the possible introduction of mechanical ventilation at the gym because
lack of adequate ventilation was suggested to be the dominant threat. The possibility of upgrading to a mechanical ventilation
system rather than the existing, passive ventilation system is explored. ETA facilitates a comparative ‘what if’ type analysis,
with the risk reduction implemented by each barrier based on the qualitative scale used in the ‘attending the gym’ scenario.
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Barrier (3) – Physical
distancing at 2 m
Barrier (2) - Occupancy
controls
Barrier (1) - VentilationInitial risk of infection Risk of Infection
Implemented
Not implemented
Implemented
Not implemented
Passive ventilation
Mechanical ventilation
Implemented
Not implemented
Implemented
Not implemented
Implemented
Not implemented
Implemented
Not implemented
Very low
Medium
Very high
Medium
Low
Low
High
Medium
Figure 8: Event tree analysis used to compare risk for different risk control measure strategies
It is important to note that the authors have made judgments on the initial relative risk and the effectiveness of barriers used in
the ‘attending the gym’ scenario based on data where possible. However, the authors are not ‘experts’ in the subject matter of
epidemiology. Judgements made are solely to show how the WBTD methodology can be used to assess Covid-19 scenarios
rather than to present a definitive risk assessment of the ‘attending the gym’ scenario. Further, all aspects of the risk assessment,
including those discussed here, should be reconsidered as new information or data become available.
Create a weighted BTD
Weighting of risk reduction as determined according to the prior description is applied to the threat pathways and consequence
pathways in the simplified BTD. Visualisation options considered were varying line thickness, varying colours, or both. An
example diagram showing colour and line thickness is presented in Figure 9.
The barrier effectiveness is considered to implement a level of risk reduction proportional to its estimated effectiveness. Hence
the line thickness to the immediate right of each barrier is modified accordingly using the 7-point scale shown in Figure 9.
Where the % effectiveness is not known with any reasonable degree of certainty then, for demonstration purposes, the line
thickness is reduced by say 1 or 2 points on the qualitative scale chosen for the risk assessment dependent based on, for
example, expert judgement.
Uncertainty of data relating to the SARS-CoV-2 virus transmission and disease progression is high at the present time, as is
the uncertainty regarding the effectiveness of each barrier. Therefore, a 5-point scale has been applied with 5 being highly
certain and 1 being highly uncertain. This is depicted as between 1 to 5 yellow stars on each barrier Figure 9. This is only
indicative of how uncertainty could be represented visually within the weighted BTD. Work will be required to further assess
interpretation of features on a WBTD and its general intuitiveness as a risk communication technique. It is acknowledged this
is likely to result in amendments to the work presented here. It should be noted that the risk and uncertainty represented in the
diagrams in this paper are for illustration purposes only and should not be interpreted to be a statement on the risk, uncertainty
of risk, or barrier effectiveness relating to Covid-19.
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Figure 9: Weighted BTD showing a risk assessment of the individual ‘attending the gym’ Covid-19 scenario with qualitative
scale representing risk
Risk assessment outcomes
This paper has discussed aspects of the WBTD methodology, including the supporting techniques, with discussion of how
they could be applied. This section of the paper discusses the application of the WBTD methodology to the ‘attending the
gym’ example scenario.
In this application of the WBTD methodology to a Covid-19 scenario, public health measures are considered, such as
vaccination, self-care and medical treatment. However, if an assessment of this example scenario was completed to fulfil an
employer’s duties under the Health and Safety at Work Act it would only need to consider how workplace activities and
barriers could contribute to prevention of the top event occurring. Employers should not generally rely on public health
measures, such as vaccination, as risk controls to meet their obligations under health and safety law. It is important to note,
use of the WBTD methodology on other hazards may require an employer to consider both prevention and mitigation.
WBTD Left Hand Side (LHS) assessment summary
The ‘presence in a poorly ventilated environment’ is according to EMG-SAGE 2020 a significant threat in terms of spreading
a virus at indoor spaces such as a gym. The authors used their subjective opinion to treat this as the dominant threat for the
‘attending the gym’ scenario and weighted the threat likelihood to be significantly higher than that of the other four threats
considered in this risk assessment.
Taking the threats grouped according to the number of barriers associated with each in turn, from threat 1 to threat 5, and
working through the assessment from threat to the top event occurring the following is noted:
• Threats 1 and 2 are the ‘use of individual shower cubicles’ and ‘use gym equipment’ respectively. The single barrier
considered for these threats is ‘surface cleaning’ and is suggested by the authors to reduce the risk from low to very
low in this assessment.
• Threats 3 and 4 are ‘use of locker’ and ‘contact with ‘doors and turnstile at the gym’ respectively. There are two
barriers associated with this threat, namely the ‘surface cleansing’ and ‘hand sensitization/clean’ that reduce the risk
from low to extremely low.
• Threat 5 is the ‘presence in a poorly ventilated environment’. There are two barriers, namely ‘occupancy control’
and ‘physical distancing’ respectively. These two barriers reduce the risk from ‘very high’ to ‘moderate’. There is
also a recommended good practice factor, namely, limited passive ventilation, for which no credit is claimed due to
uncertainty of its effectiveness at this gym.
The combined risk that the top event occurs is the summed contribution from the five threats to give an aggregated risk of
‘very high’.
WBTD Right Hand Side (RHS) assessment summary
Each top event consequence is assessed in isolation of the other consequences, this maintains the LOPA analogy of one
scenario with one outcome in terms of the assessment outcome. The starting point for the assessment of the likelihood of each
top event consequence occurring, modelled on the RHS of the WBTD, is equal to the likelihood that the top event has occurred
multiplied by the consequence pathway risk reduction factors. The likelihood of the top event occurring is stated as being
equal to the sum of the frequencies of each top event threat occurring after all the relevant risk reduction factors have been
accounted for. See Figure 9.
Individual has Covid-19,
is admitted to hospital
where they suffer fatality
due to COVID-19 as
either a primary or
secondary cause
Individual is admitted
to hospital with
COVID-19 and
recovers
Use of showers
Presence in poorly
ventilated, indoor
environment
Using gym apparatus
Contact with surfaces
at external/internal
doors and turnstile
Use of locker
Open doors,
vents and
windows
Occupancy
control c
Physical
distancing
(2 m) d
Cleaning
hands b
Surface
cleaning a
Extremely
High Very High High Moderate Low Ver y Low Extremely
Low
Attending the gym
Blood
Cancer
Individual is infected
with SARS-CoV-2
SARS-CoV-2 in
the human
population
Fatality Hospital
treatment
3
Vaccination
1
At home
care
2
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
For the Covid-19 pandemic, data shows infection with SARS-CoV-02 is not equal to the incidence of developing Covid-19 or
of subsequent death or hospitalisation from infection, even where no barriers are employed. While not shown in the ‘attending
the gym’ example, there are inherent factors which are neither by design nor implementation, such as the probability of any
particular human body to fight off an infection, that contribute to the reduction of risk of the stated consequence. It is suggested
that some contributory factors associated with certain groups or types of people can be accounted for via use of risk
modification factors (RMF) in the proposed methodology. However, the potential use of RMFs in this way would be for the
consideration of suitable experts, whom it is envisaged, would be part of the risk assessment team that would use the WBTD
methodology.
Two consequences are considered for the purpose of this risk assessment, namely:
• Individual admitted to hospital with Covid-19 and recovers, or
• Individual admitted to hospital with Covid-19 and suffers fatality resulting from Covid-19 as primary or secondary
threat.
It is acknowledged that the consequences of a SARS-CoV-2 infection are many and varied including, but not necessarily
limited to, socioeconomic consequences and health related consequences. The health-related consequences include fatal and
non-fatal immediate consequences and/or permanent/semi-permanent health effects, some of which may not be fully
understood for some time. We have chosen to limit this example assessment to the two stated immediate consequences for
demonstration purposes.
Taking the consequences individually and working through the assessment from the top event occurring to the top event
consequence occurring, the following is noted:
• Consequence 1 ‘Individual has Covid-19 and is admitted to hospital’ is mitigated by a single barrier, namely that the
’individual is fully vaccinated’. This is suggested by the authors to reduce the risk from ‘extremely high’ to
‘moderate’. Additionally, there is an RMF, ‘individual has blood cancer’ and this is suggested to increase the
individual’s risk associated with the stated consequence from ‘very high’ to ‘extremely high’ in this risk assessment.
There are many existing health conditions that could have been included and blood cancer was used to demonstrate
how such conditions could be accounted for when using the WBTD methodology. Consequence 1 has an RGP, ‘at
home care’ prior to being admitted to hospital, which would include several factors whose effectiveness cannot
readily be determined and so no credit is claimed, but which is suggested to be beneficial to the infected individual
plus following this guidance reduces the risk of spreading the infection to others.
• Consequence 2 ‘Individual has Covid-19 and suffers fatality due to Covid-19 as a primary or secondary cause while
in hospital’ has two barriers, namely that the individual is ‘fully vaccinated’, and that the individual receives an up
to date relevant ‘hospital treatment’ regime. The authors suggest that this would reduce the risk of the infected
individual suffering fatality from ‘moderate’ to ‘low’ for the purposes of this example scenario only. Consequence
2 has an RMF that covers the same underlying health condition considered for consequence 1 and results in the same
suggested increase in risk. Consequence 2 has the RGP ‘at home care’, which has already been discussed above in
relation to mitigation of consequence 1. Therefore, the likelihood that the infected individual would suffer fatality
in this risk assessment is suggested to be ‘extremely low’.
Discussion
This paper describes the application of the WBTD methodology by the authors to a fictitious example Covid-19 scenario,
‘attending the gym’. The following outcomes were delivered and described in this paper:
• For the top event defined, its threats and consequences, the potential preventative and mitigative measures – referred to
collectively as barriers – were identified.
• For each barrier, a failure analysis was performed to determine what could prevent the barrier working effectively.
Supporting techniques, such as FTA and ETA, have been used in the WBTD methodology to assess different aspects of
the barriers. For each relevant barrier, at least one degradation factor (failure mode) was identified, and an associated
recovery mechanism was identified.
• The potential contribution of underlying risk factors in the risk assessment outcome have been presented. The
methodology has included what the authors term, risk modification factors (RMF), that are not implemented barriers, but
are included in a risk assessment to better reflect the actual risk. These could be considered analogous to LOPA
conditional modifiers but with a wider scope. In the Covid-19 example scenario, identified RMFs included, pre-existing
health conditions, age and sex of a person who has Covid-19.
• The Covid-19 example assessment utilised ‘recommended good practice’ (RGP). RGPs are not credited in the risk
assessment because their potential uptake and contribution to risk reduction is likely unpredictable and hence any credit
claimed is uncertain. However, RGP can indirectly contribute to reduce risk. An example of relevant good practice
considered in the example Covid-19 scenario was opening doors, windows and vents to improve passive ventilation in a
room where people might gather.
• For the example Covid-19 top event, the WBTD methodology presented pictorially the notational flow of risk between
top event threats, consequences, and barriers in a way that the authors believe can be readily explained to both domain
experts, risk assessment specialists and non-technical people, based on discussions with colleagues and third parties.
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
• Additionally, supporting techniques have been used in this work to compare the effectiveness of different barriers. A
visual representation of the benefit of implementing each barrier and of implementing combinations of barriers was
presented in a WBTD, see Figure 9.
• The analysis of the WBTD enabled identification of the dominant top event threat, thus aided discussion of targeted
barriers and noted issues that could either positively or negatively impact on their effectiveness.
In applying the methodology, the authors identified areas where further work is required, and these included:
• Application of the WBTD methodology to non-Covid-19 top events.
The BTD method and supporting techniques discussed in this paper are used in industry sectors where accepted numerical
data exists for factors considered in a risk assessment, such as equipment failure rates and human error probability. The
outcomes of a numerical assessment could be applied to a Weighted BTD to produce a visual representation of risk and
hence risk reduction implemented by the barriers considered.
While not explored in this paper, it is suggested that this methodology would be equally applicable to other pandemics or
similar health related top events.
It is anticipated that this methodology would be suitable for hazards that can be formulated in terms of a well-defined top
event with identifiable distinct threats and consequences. The authors suggest that the WBTD methodology could be
applied to top events in different industry sectors such as, petrochemical, aviation, space, rail, and utilities. This suggestion
is based on the methodology’s use of well-established techniques such as BTD analysis, FTA, ETA and FMEA.
Widespread further work in terms of applying the methodology to other types of top events would help develop, refine,
and improve the methodology.
It is suggested that each sector, where the WBTD methodology could be applied, could utilise different supporting
techniques, i.e. techniques that they are familiar with to explore barrier effectiveness as well as their potential failure
modes and recovery mechanisms. This would contribute to the widespread useability of the WBTD methodology.
• How to better account for human performance issues, such as estimating human reliability.
Issues such as how to assess and improve human performance for many of the identified barriers has been considered in
this initial work and Covid-19 example. The supporting risk assessment techniques used in this work, such as FTA, ETA
and FMEA, form the basis of some human reliability assessment methods, see Oshiro 201), Gerben 1985 and Van
Leeuwen 2009. However, this work does not delve deeply into assessing the contribution of human factors using human
factors techniques, such as the human error assessment and reduction technique (HEART) or hierarchical task analysis
(HRA), all of which could be the subject of further investigation.
There appears to be sufficient information available about how SARS-CoV-2 can be transmitted and how to manage
Covid-19 outcomes to be able to consider human performance on risk reduction and to include both the human error and
compliance behaviour aspects of human performance, see EMG-SAGE 2020, CDC 2021. It is anticipated that significant
benefit to the risk assessment would be realised through human factors specialist involvement as part of the risk
assessment team, specifically in determining the degradation factors and hence subsequent controls, even if we cannot
yet determine with certainty the resulting percentage change in the barrier effectiveness. An area of future work would
be to consider the integration of human factors techniques with the WBTD methodology and how this supports the
primary objectives of the methodology.
• How to account for a lack of suitable data for the effectiveness of barriers.
Regarding the Covid-19 scenario assessed as part of methodology development process, a lack of suitable data was
identified for both the top event threat frequencies of occurrence and for the effectiveness of the barriers. In the absence
of suitable data for use in the example scenario, the authors developed a 7-point qualitative scale, see table 5, that was
used as a measure of relative risk (as shown in Figure 9) and a 5-point scale to indicate the uncertainty associated with
the perceived effectiveness of the barriers considered in the risk assessment. The scale used and selections on the scale
have not utilised experts in the field. The example scales and the selections are for demonstration purposes only. Further
development of suitable qualitative measures of risk or risk reduction are topics for further work and would involve
capturing consensus in some meaningful way for the risk assessment being performed. It is perceived that the
determination of qualitative scores associated with risk reduction and perceived uncertainty of barrier effectiveness could
be determined on a case-by-case basis using expert and/or stakeholder input.
While this has been a challenge in the example given in this paper, a lack of suitable data is a common issue in many
areas utilising risk assessment. Good risk assessment practitioners will always reflect on the quality of the data used, how
this influences the risk assessment outcome and ultimately what controls should be put in place.
It might be such that this method is unsuitable for risk assessments with particularly weak source data. Showing weighting,
even with high uncertainty, may be inappropriately interpreted by the target audience.
Further consideration could be to clearly define what types of risk the proposed methodology could or could not be applied
to. So far, the WBTD methodology has only been applied to the analysis of a single top event. The weighted BTD
methodology could be further developed to assess multiple linked top events.
• Representation of data uncertainty in the WBTD and how this might, or could, be perceived by the target audience.
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
Regardless of whether using qualitative or quantitative measures of relative risk reduction, for the risk assessment
outcomes to be meaningful to any given target audience, an indication of data uncertainty is required. For demonstration
purposes only, a 5-point scale has been applied as an indicator of the perceived uncertainty associated with the efficiency
of each barrier. In the scale used, 5 is an indication that the data used to support the effectiveness of a barrier is perceived
to have a low level of uncertainty, because for example, there may be several published independent studies that support
the data used. In the scale used 1 represents a high level of uncertainty associated with the data used, because for example
the studies performed may give mixed results. Further work is required to refine incorporation of data uncertainty in the
WBTD methodology, and particularly how it might be interpreted. The primary interest in this initial work was in showing
that uncertainty could be accounted for and depicted visually in the WBTD methodology. One way often used to address
uncertainty in LOPA is to apply a sensitivity study on most uncertain factors in the LOPA. This requires the assessor to
estimate a range of data that should encompass the uncertainties involved.
Conclusions
This paper has presented a novel holistic risk assessment and risk communication methodology, namely the WBTD
methodology. The methodology comprises established risk assessment techniques, widely used in the process industries,
augmented to support communication of the risk assessment outcomes, and used to assess a wide range of factors associated
with the top event considered.
There are three novel aspects of this proposed methodology:
1. The required detailed analysis of individual barriers to estimate their effectiveness (weighting) and their strengths
and weaknesses. The detailed barrier analysis can result in advice on how the barrier effectiveness could be
improved by suggesting recommended good practice or by highlighting poor practice that could result in a reduced
barrier effectiveness.
2. The use of the weighted bowtie diagram to communicate risk assessment outcomes. For example, presenting
visually the effectiveness of individual barriers and/or of threat /consequence pathways (strategies) to inform which
are likely to be the most effective at reducing risk.
3. How the weighted bowtie diagram is designed to illustrate weighting of risk using visualisation techniques such as
colour and line thickness on the threat and consequence pathways. This weighting can be used to depict relative
risk reduction implemented by each barrier and threat consequence pathway presented in the ‘weighted bow-tie
diagram’.
In addition to the three main novel aspects noted above, the proposed methodology is holistic in nature because it can be used
to assess all aspects of the top event, from the top event threats to its consequences, and including where relevant, detailed
analysis of the individual barriers, their failure modes and failure recovery mechanisms and recommendations for the effective
implementation of the barrier. The proposed methodology includes RMFs, that are not implemented barriers, in a risk
assessment to better reflect the actual risk. Additionally, a ‘recommended good practice’ (RGP) feature has been introduced
in the methodology.
The authors have found the inclusion of supporting techniques useful to analyse both individual barriers and barrier
combinations, or risk control strategies, used in the example Covid-19 scenario. Additionally, the authors have applied a
systematic approach to applying the supporting techniques to identify barrier degradation factors and their suggested controls
and further aided in the identification of relevant good practice associated with the implementation of the barrier and their
controls. These outcomes included not only what the risk factors are but also what measures were suggested to mitigate the
risk factors.
The methodology, including its supporting techniques, has been used to compare different risk control strategies and visually
present the outcomes of the comparison of Covid-19 risk control strategies. An outline of the methodology and its results were
presented to HSE colleagues, who remarked on the intuitiveness of the WBTD visual aspects and relative ease of understanding
of the risk assessment outcomes.
The intended audience of the risk assessment can include both risk assessment specialists, topic area specialists and other
stakeholders, including members of the public.
The paper includes an initial concept demonstration of the WBTD methodology, and its novel aspects used to assess a Covid-
19 example scenario. The authors regard this work as partly a demonstration of the applicability of the core techniques included
in the WBTD methodology such as BTD, FTA, and ETA to pandemic scenarios and partly demonstrating key novel aspects
of the methodology.
Disclaimer
This publication, and the work it describes, was funded by the Health and Safety Executive (HSE). Its contents, including any
opinions and/or conclusions expressed, are those of the authors alone and do not necessarily reflect HSE policy.
References
Anderson, D., Caulfield, M., Ramsden, M., Pettitt, G., Sarstedt, M.G., 2016, The Use of Bow Ties in Process Safety
Auditing, Hazards 26.
BS EN IEC 30010:2019, Risk management – Risk assessment techniques, London, British Standards Publications.
CCPS, 2001, Layer of Protection Analysis: Simplified Process Risk Assessment, AIChE CCPS 2001
SYMPOSIUM SERIES No.170 HAZARDS 33 © 2023 Crown copyright
CCPS, 2018, Bow Ties in Risk Management: A Concept Book for Process Safety, AIChE CCPS and EI, first edition, 2018.
CDC, 2021, Science Brief: SARS-CoV-2 and Surface (Fomite) Transmission for Indoor Community Environments, Updated
Apr. 5, 2021 [Viewed 04 March 2022]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/more/science-and-
research/surface-transmission.html
CGE Risk, 2022, Quantify your bow-tie. [Viewed 14 January 2022]. Available from:
https://www.cgerisk.com/products/quantify-your-bow-tie/
Ellermann, C., McDowell, M., Schirren, C, O., Lindemann, A, K., Koch, S., Lohmann, M., Jenny, M, A., 2022, Identifying
content to improve risk assessment communications within the Risk Profile: Literature reviews and focus groups with expert
and non-expert stakeholders. Published: April 11, 2022. https://doi.org/10.1371/journal.pone.026680
Emery, R., 2014, Operational Risk Using Bow-tie Methodology, Proceedings of the Symposium Series No. 159. Hazards 24,
2014, IChemE.
EMG-SAGE, 2020, Transmission of SARS-CoV-2 and Mitigating Measures, UK Government. [Viewed 18 January 2022].
Available from:
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/892043/S0484_Transmissi
on_of_SARS-CoV-2_and_Mitigating_Measures.pdf
GOV.UK, 2021, Face coverings: when to wear one, exemptions, and how to make your own, Updated 10 December 2021.
[Viewed 18 January 2022]. Available from: https://www.gov.uk/government/publications/face-coverings-when-to-wear-one-
and-how-to-make-your-own/face-coverings-when-to-wear-one-and-how-to-make-your-own
GOV.UK, 2022, Guidance on coronavirus (Covid-19) measures for grassroots sport participants, providers and facility
operators, Updated 14 January 2022. [Viewed 18 January 2022]. Available from: Guidance on coronavirus (Covid-19)
measures for grassroots sport participants, providers and facility operators - GOV.UK (www.gov.uk)
Freeman, A. L. J., Parker, S., Noakes, C., 2021, Expert elicitation on the relative importance of possible SARS-CoV-2
transmission routes and the effectiveness of mitigations. BMJ Open 2021;11:e050869. doi:10.1136/ bmjopen-2021-050869
Gerben, H., 1985, Analysis of Human Reliability on performing a Specific task, Delft University of Technology, Reliability
Engineering, 12, pp 63-78
HSE, 2009, Safety and environmental standards for fuel storage sites, process safety leadership group final report, HSE
Books.
Oshiro, F., 2014, Decision making using human reliability analysis, Monaco Engineering Solutions Limited, Symposium
series No 159, Hazards 24, Edinburgh may 7-9
Saunders, M. Lewis, P. Thornhill, A., 2012, Research Methods for Business Students (6th ed.).
J.F van Leeuwen, j.f. et al., 2009, Risk analysis by FMEA as an element of analytical validation, Journal of Pharmaceutical
and Biomedical Analysis, volume 50, issue 5, pages 1085-1087.
Gray, W. K., Navaratnam, A. V., Day, J., Wendon, J., Tim W R Briggs, T. W. R., 2021, NHS England. Covid-19 hospital
activity and in-hospital mortality during the first and second waves of the pandemic in England: an observational study, BMJ
journals, Thorax, volume 77, issue 11, http://orcid.org/0000-0002-9597-5446
