Content uploaded by Jock Baker
Author content
All content in this area was uploaded by Jock Baker on Mar 01, 2021
Content may be subject to copyright.
Independent Real-Time
Evaluation of UNICEF’s response
to Cyclone Idai in Mozambique,
Malawi and Zimbabwe
VOLUME I - EVALUATION REPORT
EVALUATION REPORT
INDEPENDENT REAL-TIME EVALUATION OF
UNICEF’S RESPONSE TO CYCLONE IDAI IN
MOZAMBIQUE, MALAWI AND ZIMBABWE
© United Nations Children’s Fund, New York, 2019
United Nations Children’s Fund
Three United Nations Plaza
New York, New York 10017
December 2019
The purpose of publishing evaluation reports produced by the UNICEF Evaluation
Office is to fulfil a corporate commitment to transparency by publishing all completed
evaluations. The reports are designed to stimulate a free exchange of ideas among
those interested in the topic and to assure those supporting the work of UNICEF that it
rigorously examines its strategies, results and overall effectiveness.
The contents of the report do not necessarily reflect the policies or views of UNICEF.
The views expressed in this report are those of the evaluators. The text has not been
edited to official publication standards and UNICEF accepts no responsibility for error.
The designations in this publication do not imply an opinion on the legal status of any
country or territory or of its authorities, or the delimitation of frontiers.
The copyright for this report is held by the United Nations Children’s Fund. Permission
is required to reprint/reproduce/photocopy or in any other way cite or quote from this
report in written form. UNICEF has a formal permission policy that requires a written
request to be submitted. For non-commercial uses, the permission will normally be
granted free of charge. Please write to the Evaluation Office at the address below to
initiate a permission request.
For further information, please contact:
Evaluation Office
United Nations Children’s Fund
Three United Nations Plaza
New York, New York 10017
evalhelp@unicef.org
Table of Contents
EXECUTIVE SUMMARY ............................................................................................................. 1
INTRODUCTION ....................................................................................................................... 8
1. COUNTRY AND OPERATING CONTEXT .......................................................................................... 8
2. BACKGROUND ................................................................................................................................... 11
METHODOLOGY .................................................................................................................... 18
1. EVALUATION PHASES ...................................................................................................................... 18
2. EVALUATION QUESTIONS ............................................................................................................... 18
3. MANAGEMENT AND GOVERNANCE ARRANGEMENTS ............................................................. 18
4. DATA COLLECTION, ANALYSIS AND VALIDATION .................................................................... 19
5. CONSTRAINTS AND LIMITATIONS .................................................................................................. 20
RESPONSE TO EVALUATION QUESTIONS .............................................................................. 22
KQ 1: RELEVANCE ................................................................................................................................... 22
KQ 2: ACHIEVEMENT OF RESULTS ....................................................................................................... 31
KQ 3: COVERAGE ..................................................................................................................................... 46
KQ 4: RESILIENCE AND PREPAREDNESS ............................................................................................. 49
KQ 5: COORDINATION ............................................................................................................................ 52
KQ 6: INTEGRATION OF GENDER ........................................................................................................ 56
LESSONS LEARNED ............................................................................................................... 57
CONCLUSIONS AND RECOMMENDATIONS ............................................................................ 60
Please see Volume II for the report’s Annexes.
Tables
Table 1 – UNICEF response and recovery strategy .................................................................................. 12
Table 2 – Key Questions for the RTE ......................................................................................................... 18
Table 3 – Summary of key informants and focus group discussions ...................................................... 20
Table 4 – Contributing preparedness measures in each UNICEF country office ................................ 27
Table 5 – Complementarity activities of UNICEF country offices ........................................................ 30
Table 6 – Qualitative assessment of performance by sector and country against key objectives of
UNICEF’s response strategies ..................................................................................................................... 33
Table 7 – Factors contributing to success / Factors that inhibited the response ................................. 39
Figures
Figure 1 – RTE subject, purpose and scope ................................................................................................. 8
Figure 2 – Map of affected areas ................................................................................................................... 10
Figure 3 – Funds received by appeal sector (Mozambique cyclone response) ...................................... 13
Figure 4 – Funds received by appeal sector (Malawi cyclone/flood response)..................................... 13
Figure 5 – Funds received by appeal sector (Zimbabwe multi-hazard response) ................................. 14
Figure 6 – Funds received compared with HAC requirements (Mozambique cyclone response) ..... 14
Figure 7 – Funds received compared with HAC requirements (Malawi cyclone/flood response) ... 15
Figure 8 – Funds received compared with HAC requirements (Zimbabwe multi-hazard response) 15
Figure 9 – Timeline of key milestones related to the cyclone response ................................................. 16
Figure 10 – Results of U-Report poll in Mozambique conducted in March 2019 ............................... 25
Figure 11 – Suspected cholera cases, Sofala Province (27 Mar – 2 Jun, 2019)...................................... 29
Figure 12 – Source of surge staff deployed to each country office ......................................................... 41
Figure 13 – Duration of surge deployments by country and functional area (days) ............................ 43
Figure 14 – International supply pipeline timelines (Mar 17 – Jul 31, 2019) ......................................... 44
Figure 15 – Local procurement supply timelines (Mar 17 – Jul 31, 2019) ............................................. 45
Page i
Preface
In early March 2019, southern Africa was hit by one of the worst natural disasters experienced in
recent history. The impact of Cyclone Idai in Malawi, Mozambique and Zimbabwe is estimated to
have caused 3 million people to need assistance, including 1.5 million children under 18.
Mozambique was hit the hardest, with approximately 1 million children affected. In addition, a
subsequent cyclone (Kenneth) hit the Cabo Delgado Province in northern Mozambique only six
weeks after Idai made landfall in Beira, Sofala Province, on the central coast.
On March 26, UNICEF’s Executive Director activated the L3 Corporate Emergency Activation
Procedure (CEAP) for the response in Mozambique and the L2 CEAP for Malawi and Zimbabwe.
While an inter-agency flash appeal was launched for Mozambique to request US$281.7 million
1
over three months, UNICEF in parallel launched an appeal for a total of $136.3 million for the
three affected countries ($102.6 million for Mozambique, $18.4 million for Zimbabwe and $15.3
million for Malawi) for the period March–December 2019.
Following the UNICEF evaluation policy that requires all L3 and L2 emergencies to be evaluated,
the Evaluation Office commissioned a multi-country real-time evaluation (RTE) of the emergency
response in Malawi, Mozambique and Zimbabwe. The evaluation was conducted approximately
three months after Cyclone Idai made landfall, at a point in time when affected communities were
gradually starting to recover. Key purposes of the evaluation were to reflect on the response in the
three countries; generate learning for future preparedness and response efforts to sudden-onset
crises; and support the direction and planning of ongoing recovery efforts. The evaluation also
aimed to strengthen accountability towards affected populations, partners and stakeholders in the
response.
The RTE was conducted by a team of independent consultants with specific expertise in key
sectors of the response. I want to sincerely thank Jock Baker for leading the team in these efforts,
and also thank all team members, including Adeline Sibanda, Claudia Perlongo, Carlotta Tincati,
Chrissie Thakwalakwa, Dorothy Matyatya, Gabriela Moreira da Silva, Jonas Chambule, Paul Kawale
and Roy Mutandwa for their work.
The RTE found that UNICEF was a critical contributor to government-led responses in all three
countries and one of the humanitarian agencies that mobilized as soon as the impact of the cyclone
became evident. UNICEF efforts to rapidly and effectively contain cholera in cyclone-affected
areas that are cholera-endemic were remarkable. UNICEF played a major role at the inter-agency
level, through its cluster lead responsibilities and its contributions to initial assessments and
initiatives to prevent sexual exploitation and abuse. This benefitted the wider humanitarian
community.
This evaluation also sheds light on specific areas that UNICEF should strengthen. These include
early warning, some aspects of preparedness planning and early recovery, including the more
systematic consideration of the specific needs of resettled communities. The evaluation identified
further weaknesses in information management and monitoring and pointed to processes with high
transaction costs, such as short surge deployments and lack of longer-term programme cooperation
agreements (PCAs), which in some cases hampered the effectiveness of the response. The
evaluation also shows the need for greater attention to gender and equity components, and for
1
Unless otherwise stated, all amounts shown are in US dollars.
Page ii
communities to more actively inform and participate in assistance operations.
The RTE would not have been possible without the support and engagement of the regional office
in Nairobi and UNICEF staff in the country offices of the three affected countries. This includes
UNICEF Regional Director for Eastern and Southern Africa Mohamed Malick Fall, Deputy
Regional Director Bo Viktor Nylund and Regional Emergency Advisor Pete Manfield; UNICEF
Mozambique Representative Marcoluigi Corsi and Deputy Representative Katarina Johansson;
former UNICEF Malawi Representative Johannes Wedenig and former Deputy Representative
Roisin De Burca; and UNICEF Zimbabwe Representative Laylee Moshiri and former Deputy
Representative Amina Mohammed. A special thanks also goes to the focal points who supported
the logistics and planning of the evaluation in each country office. These include Zlata Bruckauf
and Claudio de Sandra Julaia in Mozambique; Emmanuel Saka and Estere Tsoka in Malawi; and
Blessing Zindi and Oscar Tapera in Zimbabwe.
Finally, let me thank the members of the Reference Group who offered their knowledge and
guidance throughout the evaluation and helped the evaluation team deliver a quality product -
Stephane Arnaud; Pete Manfield; Tsedeye Girma; Hamish Young; Cecilia Sanchez Bodas; Ana
Cristina Matos; Blessing Zindi; Mekonnen Woldegorgis; Zlata Bruckauf; Jan Eijkenaar; Dorica
Tasuzgika Phiri; Fitsum Assefa; Patrick Lavand’homme; Ashley Wax, Anthea Moore, and Carlotta
Tincati; and my colleagues who managed this work, namely, Jane Mwangi and Urs Nagel. Many
thanks also go to Celeste Lebowitz, Geeta Dey and Dalma Rivero for their tireless administrative
support and to Yasmin Winther Almeida for her help throughout the evaluation.
George Laryea-Adjei
Director of Evaluation
UNICEF
Page iii
List of Abbreviations and Acronyms
AAP
ACF
C4D
CCC
CERF
CFM
CLTS
DoDMA
DRR
DTM
ERT
ESARO
FST
HAC
HPM
HRP
HSD
INGC
INGO
IOM
IYCF
KQ
MoE
MoH
MRA
NGO
OCV
PDM
RRM
RTE
UNICEF
WFP
WHO
Accountability to affected populations
Action contre la Faim
Communication for development
Core Commitments for Children in Humanitarian Action
Central Emergency Response Fund
Complaints and feedback mechanisms
Community-Led Total Sanitation
Department of Disaster Management Affairs (Malawi)
Disaster risk reduction
Displacement tracking matrix
Emergency response team
Eastern and Southern Africa Regional Office
Field support team
Humanitarian Action for Children
Humanitarian Performance Monitoring
Humanitarian Response Plan
Humanitarian Surge Deployment
Instituto Nacional de Gestão de Calamidades
International non-governmental organization
International Organization for Migration
Infant and young child feeding
Key Question
Ministry of Education
Ministry of Health
Multisectoral rapid assessments
Non-governmental organization
Oral cholera vaccination
Post-distribution monitoring
Rapid Response Mechanism
Real-time evaluation
United Nations Children's Fund
World Food Programme
World Health Organization
1
Executive Summary
Evaluation purpose and scope
This real-time evaluation (RTE) focuses on UNICEF’s programmatic and operational response and
preparedness in the three countries affected in 2019 by Cyclones Idai and Kenneth. The evaluation’s
purpose is to assess the appropriateness, effectiveness, coverage, connectedness and coordination of
UNICEF’s response. It is also intended to generate learning for future preparedness for and response to
sudden-onset crises, while generating recommendations for the direction and planning of ongoing
cyclone recovery efforts. The evaluation also aims to strengthen accountability towards affected
populations, partners and other key stakeholders.
Context
On 14 March 2019, tropical cyclone Idai made landfall in Mozambique and also brought devastation to
Malawi and Zimbabwe, the worst sudden-onset emergency, as measured by people affected, to hit
southern Africa in two decades. The storm is estimated to have caused more than 1,000 deaths and
displaced more than 240,000 people. A total of 3 million people required assistance in the three affected
countries, including 1.5 million children. UNICEF had country programmes in all three countries prior to
the cyclone, which positioned the organization to play a key role in the emergency response.
Methodology
The evaluation was jointly managed by UNICEF’s Evaluation Office and the Eastern and Southern
Africa Regional Office (ESARO). The evaluation was divided into three phases: inception, data collection
and analysis/reporting. The data collection phase included a desk review as well as a four-week visit to
Malawi, Mozambique and Zimbabwe and to UNICEF’s regional office in Nairobi, to collect data relevant
to the six evaluation questions listed in the terms of reference for the evaluation. The team interviewed a
total of 689 stakeholders, either face-to-face or remotely. Interviewees included representatives of
government authorities, bilateral donors, United Nations agencies and international and non-
governmental organizations. The number of interviews also includes 350 people (including, among
others, adolescents and people living with disabilities) participating in 46 focus group discussions for
members of displaced and host communities. The team conducted a desk review of policy and strategy
documents, evaluations, reviews, studies and other documents. A challenge faced by the RTE team in
assessing UNICEF achievements was the lack of outcome monitoring data. The team therefore relied to a
large extent on qualitative data collected from affected communities during the focus group discussions.
Summary of findings
Main findings based on each of the key questions (KQ) are as follows:
1. Relevance
UNICEF played a key role in inter-agency assessments, which helped ensure that the organization’s
overall emergency response strategy for the cyclones was aligned with humanitarian needs in all three
countries during the emergency phase. UNICEF’s support to the governments’ lead roles was widely
appreciated. In common with other humanitarian agencies, UNICEF’s response strategy in most sectors
concentrated on displaced populations in camps in the geographical areas that were most affected. This
resulted in delayed response and incomplete coverage in Malawi and, with the exception of health and
WASH, in Mozambique. Due to a combination of difficulties in accessing affected communities, gaps in
assessment data and preparedness planning in some sectors, relief materials were at times distributed
when community needs were already shifting to recovery.
2
2. Achievement of results
Performance varied by sector, with consistently good results achieved in all three countries in WASH,
health and nutrition. Preparedness and effective partnerships contributed to the success of the response,
including rapid and effective containment of cholera. Activation of L2 and L3 protocols resulted in
effective and timely support by the regional office (ESARO) and by UNICEF headquarters, including an
immediate allocation from UNICEF’s reserve funds to kick-start the response, early visits from senior
decision makers along with rapid deployment of surge staff sourced from UNICEF and external rosters.
Efficiency and effectiveness were hampered by the time needed for procurement that resulted in delayed
distribution of relief items. In Mozambique and Malawi, gaps in surge deployments and short durations
for many of them also had a negative impact on efficiency and effectiveness. Additionally, aside from
staff in the supply section and some staff working in WASH, there was little evidence that UNICEF staff
routinely considered cost when deciding on intervention options. Nonetheless, some UNICEF initiatives
showed the potential to be replicated in future responses with government ownership.
3. Coverage
Gaps in assessment data in Malawi and Mozambique made it difficult for humanitarian agencies to
prioritize assistance to vulnerable groups. Targeting was influenced by community leaders, which at times
resulted in allegations of favouritism and sexual exploitation. In Mozambique and Malawi, UNICEF
supported and made use of inter-agency complaints and feedback mechanisms, but none of the UNICEF
country offices had internal protocols for dealing with complaints or feedback from communities affected
by disasters, whether these complaints came from inter-agency complaints mechanisms or from other
sources. UNICEF emergency interventions initially focused on areas that appeared to have been hit the
hardest and were in the most need of humanitarian assistance, which resulted in unevenly meeting needs
in Mozambique and Malawi. Apart from examples from the WASH sector, there was little evidence of
targeting assistance to people with disabilities. In Zimbabwe, the response and coordination for child
protection was relatively quick and had good coverage, but this was not the case in the other two
countries. At the same time, UNICEF and other agencies faced a particular challenge in Zimbabwe due
to allegations of sexual exploitation and abuse arising from the military’s involvement in initial relief
operations.
4. Resilience and preparedness
UNICEF Malawi’s response plan was the sole example of an overall plan that recognized the need for
communities to restart livelihoods without waiting for large-scale recovery programmes. In Mozambique
and Zimbabwe, UNICEF focused mainly on preparations for long-term recovery, which was expected
to take time to gain momentum. In these two countries there was instead a reliance on sector-specific
phased approaches to address early recovery. Together with a certain lack of coherence, sector-based
approaches for early recovery faced challenges due to the short-term nature of funding and project
design. Cash transfer mechanisms potentially had a very important role to play in early recovery, but
Malawi was the only country where UNICEF had the necessary preparedness in place and could
implement them in a timely fashion. UNICEF’s longer-term programming in each country has been
conducive to promoting resilience. Government counterparts viewed UNICEF as a key partner, not only
in responding to this disaster but also in building their capacities during recovery and in continuing
support them in responses to future disasters.
5. Coordination
Internal coordination between the UNICEF country office and the operational hub was good in
Zimbabwe. In the two other countries, the effectiveness of coordination varied by sector and was
challenged by rapid turnover of surge staff. Information management was a constant challenge, especially
since most clusters lacked adequate information management capacities. UNICEF was viewed by the
humanitarian community as a team player and used its comparative advantage to benefit the wider
humanitarian community. Although the effectiveness of UNICEF’s coordination of implementing
3
partners and clusters varied depending on sector and country, performance was generally of good quality.
The relatively good performance of UNICEF-led clusters and working groups contributed to imbalances
vis-à-vis levels of assistance for some sectors not led by UNICEF (e.g., shelter), which had a significant
impact on children’s well-being.
6. Integration of gender
Gender analyses carried out by peer humanitarian agencies showed that the gender and equity dimensions
of the response were comparable in all countries, including gender and equity gaps. There was some
gender mainstreaming in WASH, education, C4D, health and, to a limited extent, child protection.
UNICEF country offices did not conduct gender analyses of their own programmes and there was little
evidence that systematic use of gender analyses done by other agencies had informed response
implementation or that women’s participation had significantly informed UNICEF decision-making
processes. UNICEF was a strong supporter of community-based collective call centres in Mozambique
and Malawi that were adapted to collect feedback and complaints, including complaints related to
prevention of sexual exploitation and abuse (PSEA).
Conclusions
This section on conclusions begins with an overall statement, followed by summary versions of
conclusions and then recommendations linked to each conclusion. Conclusions have been categorized
according to theme, starting with cross-cutting issues, then emergency operations and finally monitoring,
evaluation and information management.
OVERALL STATEMENT
UNICEF was a major international actor during this cyclone response and made significant
contributions to government-led responses to the emergency. UNICEF’s contributions included
collaboration with government authorities and partner agencies to ensure that cholera, which is
endemic in all three countries, did not cause a ‘double-disaster’ event. UNICEF’s achievements
could be attributed to a combination of pre-existing relationships with the government in
multiple sectors; effective inter-agency coordination; UNICEF’s willingness to advance
significant reserve funding to kick-start the response; programming that integrated preparedness
and resilience; and in-country, regional and global surge mechanisms to fill capacity gaps.
UNICEF provided key support to local governments unaccustomed to dealing with a disaster of
such a large scale to help these authorities fulfil their lead coordination roles. UNICEF fulfilled
its IASC cluster responsibilities relatively well and provided important support to inter-agency
initiatives, including PSEA systems and initial multisectoral assessments.
UNICEF should view this as a timely and good-quality response that was well organized, with
strategic and timely support provided by both the regional office and headquarters divisions.
Despite this, there were some important gaps in the organization’s response. UNICEF was well-
positioned after governments had declared an end to the emergency phase to play a critical role
in long-term recovery, but the transition period from the emergency to early recovery phases
lacked coherence in Mozambique and Zimbabwe, because sectors/clusters were mostly
operating independently, in contrast to the initial emergency phase when there was robust overall
coordination. The disaster also left women and adolescent girls more vulnerable and exposed to
sexual exploitation and increased the vulnerability of people living with disabilities. While these
risks were recognized in UNICEF strategies, there was no systematic effort to put into practice a
response to the risks based on a relevant analysis.
Emergency operations
4
1. Preparedness and anticipatory action
Preparedness planning proved to be a significant asset that strengthened UNICEF’s response in
all three countries. However, preparedness could have been improved by having ‘shadow’
response teams in different sections; setting up cash-based assistance as an intervention option;
more systematically considering early recovery in the initial emergency response to meet
community needs in real-time; and planning appropriate support for resettlement of
communities outside their places of origin.
2. Transition to early recovery
Due in large part to UNICEF’s long-term programming in multiple sectors and its strong
relationships with governments, UNICEF is positioned to play a major role during cyclone
recovery in all three countries. The major impacts of the cyclones were in areas with relatively
high levels of poverty; UNICEF’s role was to not only support cyclone recovery, but also
provide an opportunity to ‘build back better’ and raise living standards in communities.
Nonetheless, resilience could be more systematically incorporated into intervention design.
School design could be improved to accommodate potential future displacements near at-risk
areas. Scenario-planning could inform standards for WASH facilities, shelter and protection.
UNICEF, and the broader international humanitarian community, struggled to meet the needs of
displaced populations in resettlement sites that were not their places of origin. This is largely due
to the fact that preparedness planning did not consider the scenario of massive resettlement.
While this was a challenge for UNICEF in all three countries affected by the cyclones, in
Mozambique the ad hoc nature of the resettlement of more than 50,000 people absorbed a
considerable amount of resources which, along with the need to respond to Cyclone Kenneth,
made early recovery in affected areas even more difficult.
3. Value-added of surge deployments
The value of surge deployments in Mozambique and Malawi was decreased by the short
duration of deployments. The short deployments created gaps between deployments, had a
negative impact on teamwork, increased transaction costs for UNICEF management, and led to
inconsistencies in relationships with partners and communities. In Mozambique, there were
insufficient human resources surge staff to cope with the increased workload. There was also a
lack of Portuguese-speaking surge candidates to fill strategic liaison and coordination roles.
Zimbabwe provided a good-practice example of surge practices that optimized the added value
of surge deployments in terms of deployment length and demonstrated the potential for surge
staff to fill critical capacity gaps.
4. Supply chain tracking
Procuring relief items was a major component of the cyclone response. UNICEF’s Supply
Division provided effective and efficient support to emergency operations through expedited
procurement and transportation of prioritized relief items, supported by six-week deployments
of surge staff where required. The main area for improvement identified in this evaluation was
the need to improve tracking systems so that they provide a more user-friendly tool for
programme staff to track procurement in real time. Improvements in tracking will also help
improv UNICEF’s accountability to affected populations by giving a more complete picture of
the supply chain from request to delivery to the end user at the community level.
5. PCA processes in Mozambique
High transaction costs for PCA processes for some sections (notably education and child
protection) in Mozambique delayed response implementation and reduced its effectiveness. The
5
lack of longer-term PCAs to support early recovery also reduced response effectiveness, as did
the rapid turnover of UNICEF focal points, which resulted in a situation where negotiations had
to be constantly restarted.
Monitoring, evaluation and information management
6. Information management and monitoring
UNICEF’s operations and cluster coordination roles were constrained by a general lack of
relevant data during the first weeks of the response. In Mozambique and Malawi this
contributed to uneven coverage of affected populations, with the exception of WASH
interventions. This was a challenge common to all agencies, including the two countries’ national
disaster management agencies. In all three countries, UNICEF’s country offices adapted
RapidPro open source online platforms to support decision making by improving data
visualization, not only for UNICEF but for other stakeholders. However, this took time to set
up, and there was limited evidence of its use beyond UNICEF apart from in Mozambique,
where that country’s National Disaster Management Institute used it for communication
purposes.
Delays in revising the initial targets set in the Humanitarian Action for Children (HAC) appeal
and in the Humanitarian Response Plan in all the three countries reflected the lack of systematic
monitoring and tracking of changing needs. This highlights the need for specific tools fit for the
purpose. UNICEF’s monitoring has tended to report on accomplishments related to activity and
output targets, as opposed to quality of interventions or their outcomes. The RTE team preparing
this report partially attributed this to a tendency of UNICEF staff to view outcomes in terms of
behavioural changes instead of as a sequence of immediate, medium-term and longer-term
outcomes.
UNICEF also faced challenges with internal coordination in Mozambique and Malawi, notably
between capitals and hubs set up to manage field operations. Contributing factors included rapid
turnover of UNICEF surge staff, gaps in information management systems and insufficient
monitoring visits by some section heads.
7. Need to improve preparedness for RTEs
One of the challenges faced during this RTE was differing expectations about which performance
benchmarks should be used to measure performance. The differences in expectations were
rooted in the evaluation TOR, which made reference to the HAC, the CCCs, Country Response
Strategies and sector strategies. Several international humanitarian agencies who routinely conduct
real-time evaluations use performance measurement templates with a scoring system that is
contextualized for a specific disaster response. UNICEF could develop a suitable template based
on the revised CCCs as part of institutional emergency preparedness, together with guidance on
contextualizing indicators for scores. A more standardized approach to RTEs has a number of
potential advantages: streamlining RTE processes, clarifying expectations about how performance
is measured and making it easier to compare different RTEs to help identify institutional gaps,
facilitate institutional learning and reinforce accountability.
Cross-cutting issues
8. Accountability to affected populations (AAP)
UNICEF can point to some areas of progress towards meeting its AAP commitments, notably
support of inter-agency PSEA initiatives and protection-related activities in child protection and
health (e.g., prevention of sexual and gender-based violence) in all three countries. However,
findings indicate that affected communities have largely been passive recipients of assistance.
6
Community members in focus groups were unaware of what assistance was being delivered, and
their participation in assistance operations has been marginal. UNICEF support to inter-agency
complaints and feedback mechanisms in Mozambique and Malawi was consistent with good
practice, although none of the three UNICEF country offices had internal protocols for
following up on complaints or feedback from communities affected by disasters, whether this
feedback was received from inter-agency complaints and feedback mechanisms or from other
sources.2 While there was considerable interest among staff in improving AAP, there was
relatively little knowledge about how to achieve this in practice. On the other hand, many of
UNICEF’s international NGO partners have made considerable progress integrating AAP into
their humanitarian programming. This offers potential learning opportunities for UNICEF.
9. Gender and equity
UNICEF did not carry out a gender analysis of its response in any of the three countries. This,
along with gaps in sex- and age-disaggregated data, resulted in limited attention to gender and
equity components, limited engagement of women and girls in decision making and lack of
monitoring of gender gaps.
10. Value for money of UNICEF interventions
Studies have shown that achieving value for money (VFM) relies on having the right systems and
processes in place, which, coupled with appropriate incentives, encourage application of VFM
principles when managing projects, programmes and resources. An important step in promoting
value for money is shifting mindsets and organizational culture so that staff are aware of their
responsibilities as ‘investment managers’. As such, they have the authority to make decisions on
behalf of communities affected by humanitarian crises to optimize the value for money of this
assistance. With the exception of the procurement unit, where it was clear that VFM is
embedded in systems and ways of working, this RTE team observed relatively low awareness of
or analysis using VFM among UNICEF staff. This is an indicator that cost effectiveness was not
being optimized.
Summary of recommendations
Ten recommendations based on evidence and conclusions related to this evidence are targeted at
UNICEF headquarters, the Eastern and Southern Africa Regional Office, UNICEF’s Evaluation Office
and UNICEF country offices. Timeframes for implementing these recommendations are defined where
appropriate. A summary of the recommendations is as follows:
R1. Improve early warning and preparedness planning based on lessons learned and include thresholds
that trigger anticipatory actions prior to the disaster.
R2. Organize workshops, involving field-based staff from UNICEF and partners, to update
assessments and guide the transition between humanitarian response and recovery to ensure that
UNICEF is optimizing its value-added for affected communities.
R3. Ensure that surge deployments, notably by UNICEF staff, optimize the value-added of UNICEF’s
response and recovery interventions.
2
While collective complaints and feedback mechanisms are considered good practice, they work most effectively when they can
interface with agencies’ own internal systems so that accountability and protocols are clear. See, for example, Inter-Agency
Standing Committee, Guideline: Inter-agency community-based complaint mechanisms –Protection against sexual exploitation and abuse, 2016.
7
R4. Improve the efficiency of the supply chain through more effective use of regional resources and
support UNICEF’s accountability to affected populations by using innovative approaches to
extend tracking to communities affected by disasters.
R5. Improve protocols for incorporating standby PCAs into preparedness planning to streamline
processes consistently in all sectors during future responses.
R6. Improve accountability to affected populations (AAP) to optimize the value-added of UNICEF
operations for disaster-affected communities and fulfil UNICEF’s accountability commitments.
R7. Strengthen gender integration in emergency preparedness, response and recovery phases, across
planning, implementation and monitoring and evaluation activities.
R8. Promote greater value for money (VFM) awareness among staff and partners while improving
systems to facilitate VFM analysis by UNICEF decision makers.
R9. Strengthen information management and monitoring systems to guide UNICEF interventions and
more clearly demonstrate outcomes, including at the level of the affected community.
R10. For future RTEs, UNICEF should develop and field test a template and associated guidelines
based on the revised CCCs. The template should use indicators that are developed during the
inception phase of an emergency response and adapted to the operating context. These should be
part of the toolkit supporting UNICEF’s preparedness for corporate emergencies.
8
Introduction
1. This is the report of the multi-country, real-time evaluation of UNICEF’s response to Cyclones Idai
and Cyclone Kenneth from March to July 2019. Figure 1 summarizes its subject, purpose and scope
of the evaluation.
Figure 1 – RTE subject, purpose and scope
2. This evaluation focuses on UNICEF’s programmatic and operational response and preparedness in
the three countries (Mozambique, Malawi and Zimbabwe) affected by Cyclone Idai.
3
Building on
evidence of how UNICEF responded to these emergencies, the evaluation’s purpose is to generate
strong learning for future preparedness and response efforts to sudden-onset crises, while generating
recommendations for planning and directing ongoing recovery efforts in the three countries. The
evaluation also aims to strengthen accountability towards affected populations, partners and
stakeholders. The terms of reference (TOR) of the evaluation provides further detail (see Annex 1).
3. Fieldwork for the RTE took place over one month (June 17 – July 16) as follows: Malawi (June 17–
22), Mozambique (June 23 – July 6) and Zimbabwe (July 7 – 13). Three members of the evaluation
team also travelled to Nairobi (July 15 – 16) for interviews with staff from UNICEF’s Eastern and
Southern Africa Regional Office (ESARO) who had been involved in the emergency response. Given
the real-time nature of the exercise, the summative aspects of this RTE cover the response up to the
point when the evaluation team was deployed to the field.
1. Country and operating context
3
In Mozambique, UNICEF’s response to Cyclone Kenneth, which hit Cabo Delgado at the height of the response to Cyclone
Idai, was also covered by the RTE.
9
Operating context
4. On 14 March 2019 a tropical cyclone (‘Idai’) made landfall in Mozambique, affecting Mozambique,
Malawi and Zimbabwe and causing what is considered the worst sudden-onset emergency to hit
southern Africa in two decades. The impact is estimated to have caused more than 1,000 deaths and
displaced more than 240,000 at the height of the crisis. Three million people required assistance,
including 1.5 million children under 18. Mozambique was hit the hardest, with an estimated 1.85
million people requiring assistance (including 1 million children).
4
Communities were initially
displaced to 161
5
transit centres; approximately half of the yearly agricultural production was
destroyed; and more than 50 health facilities and 3,500 school classrooms were damaged.
5. The catastrophic effects of the tropical depression which caused Cyclone Idai came in waves, with
the cyclone’s landfall in Beira only following a series of heavy rains and flooding in the provinces of
Zambezia and Tete (Mozambique) and in the southern districts of Malawi. On March 8, the
President of Malawi declared a state of disaster. On March 11 in Mozambique, the government
recommended activating the ‘Institutional Red Alert’ to enable full operation of the emergency
coordination mechanism. The following day the United Nations humanitarian country team (HCT)
convened a meeting to take stock of the situation. Following its landfall on March 14th in Beira
(Sofala Province), Cyclone Idai progressed inland, causing torrential rains in eastern parts of
Zimbabwe (Chimanimani, Nyanga and Chipinge districts) and in central Mozambique. UNICEF
country offices in all three countries were already responding to a drought that had increased the
vulnerability of populations.
6. The scale and the impact of Cyclone Idai varied significantly across the three countries. In
Zimbabwe, only two districts were seriously impacted, whereas in northern Mozambique, large
areas of five provinces were affected by the two cyclones that struck within six weeks of each other
(Cyclone Kenneth hit in April). The RTE nonetheless identified common contextual features in these
countries, which are all prone to recurring climate-related emergencies. Figure 2, below, illustrates the
affected areas in each of the three countries and offers a snapshot of the categories and numbers of
affected people.
4
UNICEF Mozambique Cyclones Idai and Kenneth Situation Report #11 (1-10 June 2019).
5
UNICEF Mozambique HAC 2019.
10
Figure 2 – Map of affected areas
Source: UNICEF (2019) Cyclone IDAI and Kenneth post-impact situation May 2019 - infographic
Mozambique
7. Cyclones and floods are recurring hazards in Mozambique. Cyclone Favio (2007) and Cyclone Eline
(2000) – along with the history of flooding of the Zambezi River – illustrate the natural disaster risk
to the country.
6
Mozambique’s National Disasters Management Institute (INGC) has gained broad
respect for its ability to respond to disasters,
7
but Cyclone Idai was exceptional in that it was the first
6
Cosgrave, John, et al., Inter-agency real-time evaluation of the response to the February 2007 floods and cyclone in Mozambique, IASC, 2007.
7
Ibid.
11
cyclone to hit a major urban centre (Beira), and because so many people were affected. The cyclone
was also notable for the extent to which it inflicted broad damage on infrastructure, agricultural
production and livelihoods. Just six weeks after the landfall of Cyclone Idai, Cyclone Kenneth struck
the northern province of Cabo Delgado, which put even more strain on the country’s resources. The
effects of these two cyclones were aggravated by pre-existing conflict in the north and ongoing
drought, and by the general vulnerability of the population – and specifically children – to
malnutrition and HIV and AIDS.
8
As of July 2019, an estimated 67,500 children (aged 5–59 months)
were acutely malnourished and in urgent need of treatment due to the combined effects of the years-
long drought and the effects of the cyclones.
9
Malawi
8. Southern Malawi is prone to regular emergencies caused by floods, cholera outbreaks and prolonged
lean seasons. Malawi experienced an even more devastating flood emergency during 2015 in virtually
the same areas affected by Cyclone Idai in 2019.
10
In March 2019, Malawi had just received Central
Emergency Response Fund (CERF) funding for the lean season when Cyclone Idai hit.
11
Heavy rains
had been affecting the southern and central parts of the country a month prior to the President’s
declaration of a state of disaster on March 8. Lessons learned from 2015 informed the 2019 response
by Malawi’s Department of Disaster Management Affairs (DoDMA) and UNICEF.
Zimbabwe
9. Cyclone Idai also hit Zimbabwe, a country more prone to slow-onset natural emergencies. The
cyclone’s effects, though severe, were limited in geographic scope but nonetheless added to existing
humanitarian challenges brought about by economic and drought-related shocks, which had already
made an estimated 5.5 million people food insecure.
12
Access to food was hampered by soaring food
prices (50 –150 per cent above the five-year average) and impact on livelihoods of reform efforts
meant to stabilize the economy.
13
2. Background
10. The Cyclone made landfall on March 14. On March 26, following activation of the scale-up in
response for Mozambique by the Inter-Agency Standing Committee (IASC), UNICEF’s Executive
Director activated the L3 Corporate Emergency Activation Procedure (CEAP) for the response in
Mozambique and the L2 CEAP for Malawi and Zimbabwe. Shortly after the activation of the
CEAP and, in line with UNICEF’s evaluation policies, the UNICEF Evaluation Office
commissioned a real-time multi-country evaluation covering the three countries, an evaluation jointly
managed with the Eastern and Southern Africa Regional Office.
Planning the response
11. UNICEF’s response and recovery planning were comparable in all three countries, with the main
differences being the relative scale of disaster impact. UNICEF had an overall goal of mobilizing a
multisectoral response led by respective governments to meet the humanitarian needs of affected
populations. Key elements of UNICEF’s strategies are described in Table 1, below.
8
UNICEF, UNICEF Mozambique Country Office Annual Report, 2017.
9
IPC Acute Food Insecurity Analysis, April 2019 – February 2020, Issued on July 25, 2019.
10
Wasunna, N. and E. Tsoka, Internal Lessons Learned from Malawi Floods, 2015.
11
Start Network/Acaps, Briefing Note – 12 March 2019: Malawi Floods.
12
Zimbabwe Vulnerability Assessment Committee (ZimVAC), 2019 Rural Livelihoods Assessment.
13
OCHA, Zimbabwe: 2019–2020 Humanitarian Appeal Revision (February 2019 – April 2020); IPC, Zimbabwe: Acute Food
Insecurity Situation February – May 2019.
12
Table 1 – UNICEF response and recovery strategy
Programme
Coordination
Provide immediate support in WASH,
health, nutrition, child protection and
education
Support government agencies, who are primary
providers of services and duty bearers, as they
provide and coordinate the international
response
Implement an approach that equitably
reaches affected communities in different
locations
Lead and coordinate the cluster response in
sectors where UNICEF is cluster lead or co-lead
Support vulnerable groups in hard-to-reach
rural areas
Seek and foster inter-agency complementarity by
implementing the response strategy in
coordination with key partners
Prioritize and addresses needs of vulnerable
groups, including girls; children with
disabilities; those living with HIV and AIDs
Operations
Use a programming approach that delivers
and integrated package of services
Reinforce UNICEF’s field presence and
humanitarian capacity to provide operational
support
Link UNICEF’s humanitarian response to
recovery programming through working
with governments and the cluster system
Establish operational hubs in the vicinity of the
areas most affected
Implement ‘building back better’
programming to build community resilience
Promote innovation through technological
platforms and approaches for assessment, data
collection, monitoring, information sharing and
collecting feedback
12. At the end of March, while launching the UN Flash Appeal for Mozambique, the revision of the
Flash Appeal for Zimbabwe and the Flood Response Plan and Appeal in Malawi, UNICEF revised
its Humanitarian Action for Children (HAC) plans for all three countries. The revised HACs
amounted to a total of $136.3 million, including $102.6 million for Mozambique, $18.4 million for
Zimbabwe and $15.3 million for Malawi. Altogether, the plans aimed to reach 1.6 million people,
including 854,000 children, from March to December 2019.
14
The HACs identified specific targets
and related plans to guide UNICEF’s response. Annexes 3 and 4 provide consolidated details of
UNICEF’s programme targets and achievements as of mid-2019.
15
13. The field mission for this RTE took place after governments in each country had ended the
emergency phase of the response and the needs of the majority of affected people had transitioned to
early recovery. Substantial humanitarian needs nevertheless remained.
16
The RTE was therefore able
14
$8.26 million for the flood response in Malawi (SitRep 22-29 March) and $10.9 million for the Cyclone response in Zimbabwe
(SitRep 25-31 March).
15
UNICEF, Cyclone IDAI and Kenneth post-impact situation, May 2019.
16
IASC, Operational Peer Review: Mozambique Cyclone Idai Response, 2019.
13
to both assess and capture learning relevant to UNICEF’s response and at the same time inform the
transitional phase.
Operational context
14. The governments of Zimbabwe, Malawi and Mozambique declared national emergencies on
March 8, March 17 and March 19, respectively. Cyclone Kenneth struck Cabo Delgado Province in
northern Mozambique six weeks later on April 27 – 28.
15. As of mid-2019, UNICEF had received a total of $52.2 million for the response.
17
As illustrated in
Figures 3, 4 and 5, WASH absorbed the largest proportion of funding (30–54 per cent) in both
Mozambique and Zimbabwe and was second to nutrition (which received 36 per cent) in Malawi.
Health received between 14–17 per cent of emergency response funds in the three countries.
Education and child protection received less than 12 per cent of all funds.
18
Figure 3 – Funds received by appeal sector (Mozambique cyclone response)
Figure 4 – Funds received by appeal sector (Malawi cyclone/flood response)
17
This amount does not include in-kind support received by UNICEF, including transportation and staffing support received
through standby deployments According to data provided by UNICEF Office of Emergency Programmes, staffing support
amounted to $1,579,000 in standby in-kind support at the time of the RTE field visit.
18
In Zimbabwe, funds for education amounted to only 1 per cent of the total received.
14
Figure 5 – Funds received by appeal sector (Zimbabwe multi-hazard response)
16. Figures 6, 7 and 8 show funding received as of the end of June 2019 (Zimbabwe)
19
and end of July
2019 (Mozambique and Malawi)
20
compared with the funding requirements. They also show the
breakdown of funds received by sector.
17. As of July 2019, UNICEF’s cyclone response in Mozambique had received $33.5 million in funding,
approximately one third of HAC funding requirements.
21
Figure 6 – Funds received compared with HAC requirements (Mozambique cyclone response)
19
UNICEF Zimbabwe, Zimbabwe Humanitarian Situation Report: Multi-hazard, mid-year sitrep – 30 June 2019.
20
UNICEF Mozambique, Mozambique Humanitarian Situation Report: Cyclone response mid-year Sitrep No. 12 (March – July
2019).
21
At the time of the RTE field mission, the response plan and funding requirements were in the process of being revised.
15
18. UNICEF Malawi secured $10.2 million (or more than two thirds) of the $15.3 million required in the
2019 humanitarian appeal for the cyclone and flood response in that country.
Figure 7 – Funds received compared with HAC requirements (Malawi cyclone/flood response)
19. By mid-2019, Zimbabwe’s multi-hazard response received $8.5 million (just under half) of the $18.4
million requested to meet the increasing humanitarian needs caused by the drought and the cyclone.
Figure 8 – Funds received compared with HAC requirements (Zimbabwe multi-hazard
response)
16
Timeline
20. A timeline showing key UNICEF-specific (shaded in grey) and external events relevant to the
humanitarian crisis and response is shown in Figure 9, below.
Figure 9 – Timeline of key milestones related to the cyclone response
Key UNICEF internal events
Key external events
Month
MAL
Heavy rains and flooding in southern districts of Malawi; President of Malawi declares
state of disaster (March 8th)
MARCH
MOZ
Heavy rains and flooding in Mozambique (Zambezia Province and Tete Province),
followed two weeks later by Cyclone Idai, which makes landfall in Beira (Sofala
Province) (March 14th)
ZIM
Cyclone progresses inland, causing torrential rains in eastern Zimbabwe
MOZ
IASC activates system-wide scale-up for Mozambique for three months (March 22nd);
Clusters activated
MOZ
By March 22, 90 per cent of Beira/Dondo water supply system is restored, providing
access to clean water for 340,000 people
ALL
UNICEF activates L3 CEAP for Mozambique for three months (March 26th) and
activates L2 for Zimbabwe and Malawi
ALL
UN Flash Appeal launched in Mozambique (March 27th); Revised Flash Appeal in
Zimbabwe; Republic of Malawi Flood Response Plan and Appeal
ALL
UNICEF Humanitarian Action for Children ('HAC') appeals launched for all three
countries (March 27th)
MOZ
UNICEF oral cholera vaccine (OCV) campaign starts; hubs established in Beira and
APRIL
17
Key UNICEF internal events
Key external events
Month
Chimoio; logistics base in Beira set up and warehousing capacity established
MOZ
Last day of emergency operations centre at the airport (April 25th) – move towards
provincial coordination mechanisms
MOZ
Concern over new cyclone headed north towards Cabo Delgado; UNICEF team
deployed in advance and initial supplies sent to Pemba
MOZ
Cyclone Kenneth makes landfall in Cabo Delgado (April 26th)
MOZ
Child Health Week (May 6) in the 21 most affected districts of Sofala, Manica, Tete and
Zambezia provinces
MAY
ZIM
Official re-opening of schools affected by flooding (May 7)
MOZ
HCT discusses scale-down of OCHA (May 14)
MOZ
Cyclone Kenneth three-month humanitarian response plan finalized
MOZ
OCV campaign starts in Cabo Delgado (May 16)
MAL
Elections in Malawi (May 21)
ZIM
2nd round of OCV (May 27)
MOZ
Operational Peer Review (OPR) (May 27–31)
All
Governments declare end of emergency phase
MOZ
Government pledging conference on recovery (May 30–31)
MOZ
Scale-up/L3 expires/deactivated
JUNE-
JULY
MOZ
UN Secretary-General visits Mozambique
ZIM
World Bank Board meeting on recovery in Zimbabwe
18
Methodology
1. Evaluation phases
21. The evaluation was divided into three phases: inception, data collection and synthesis. Key milestones
during the evaluation process were the inception report and debriefings at the end of each country
visit and at the UNICEF regional office in Nairobi on 16 June 2019.
2. Evaluation questions
22. This evaluation aimed to draw evidence-informed conclusions based on OECD/DAC evaluation
criteria of appropriateness/relevance, efficiency, effectiveness, coordination, connectedness and
coverage from the perspective of UNICEF’s Core Commitments for Children in Humanitarian
Action.
23. Key questions (KQ) in the inception report for this evaluation are shown in Table 2, below. The KQ
are based on the evaluation terms of reference while considering relevant findings from the inception
phase. The team developed an evaluation matrix for these evaluation questions; the matrix outlined
potential sources of evidence and indicators to guide data collection and its subsequent analysis. The
matrix allowed the evaluation team to organize data and information and make it easier to build a
chain of evidence from findings to conclusions to recommendations.
Table 2 – Key Questions for the RTE
Key
Question
number
Theme
Evaluation
criteria
KQ 1
How appropriate is UNICEF’s response strategy (present and planned) in
reaching the most affected populations?
Relevance
KQ 2
To what extent has UNICEF achieved/is UNICEF achieving its intended
results, and within the planned timeframe (consider HAC, response plans,
adherence to CCCs, etc.)?
Efficiency
Effectiveness
KQ 3
To what extent was the affected population, including persons with disability,
adequately identified, targeted and reached by UNICEF and its partners?
Coverage
Effectiveness
KQ 4
To what extent is UNICEF’s response contributing to longer-term goals of
enhancing prevention of future emergencies, mitigation of negative effects of
future natural hazards (resilience/sustainable solutions) and preparedness?
Relevance
Effectiveness
Connectedness
KQ 5
How effectively and efficiently has UNICEF coordinated its response both
internally and externally (with key actors including other UN Agencies, CSOs
and developing partners, national and local governments)?
Relevance
Effectiveness
Coordination
KQ 6
To what extent was gender integrated into needs assessments, planning,
implementation, monitoring and reporting of the response, as well as in
recovery planning?
Relevance
Effectiveness
3. Management and governance arrangements
24. The RTE was jointly managed by UNICEF’s Evaluation Office and ESARO in line with UNICEF’s
evaluation policies, which stipulate that the Evaluation Office and regional offices are responsible for
19
the management of evaluations of L3 and L2 emergencies, respectively. Managers were supported by
a Reference Group composed of representatives from EMOPS, Programme Division, Supply
Division, ESARO and the country offices affected by Cyclone Idai. The three country offices
provided relevant support to the RTE team, including appointing a focal point for this evaluation.
Additional details on management and governance arrangements are provided in the TOR for this
RTE (see Annex 1).
25. During field visits, the team collected additional documents from partners, including country and
provincial strategies, assessment reports, monitoring reports, presentations, evaluations and reviews
of lessons learned.
26. Emerging conclusions were discussed with stakeholders during an inter-agency validation workshop
in Mozambique and during debriefings with UNICEF staff at each country visit, and subsequently
with ESARO and headquarters staff. To fulfil the ‘real-time’ objectives of this evaluation, a series of
immediate recommendations was developed in consultation with UNICEF country offices; these
recommendations were reflected in the ‘real-time’ action plans shared with UNICEF immediately
following field visits (see Annex 2).
4. Data collection, analysis and validation
27. The RTE used a mixed-methods approach to data collection, starting with an inception phase that
included a desk review and preliminary interviews with selected key informants. During the data
collection phase, the team conducted field visits to Malawi (June 16–22), Mozambique (June 23–
July 6), Zimbabwe (July 6–13) and to UNICEF’s Regional Office in Nairobi (July 14–17).
28. Key informants were purposefully selected based on stakeholder mapping done during the
evaluation’s inception phase. In addition to interviewing the 120 UNICEF staff engaged in the
response at either field, country, regional or headquarters levels, the evaluation team also interviewed
219 other stakeholders (drawn from implementing partners, governments, UN agencies, donors, and
international and national NGOs).
29. Communities affected by the emergency represented a primary source of information for the RTE. A
total of 350 community members contributed inputs for this RTE during 52 focus group discussions
in the three countries. An average of six people participated in each focus group. Participants in the
focus group discussions were selected to achieve a balanced representation based on geographical
location and their status as displaced, resettled and returnee populations. Selection of participants was
also affected by logistical and security factors. Approximately one third of the focus groups were
conducted with adolescents and youth aged 14–21. Some 68 per cent of focus group participants
were women or girls. It was initially planned that separate focus group discussions would be held with
people with disabilities but, due to time constraints, apart from two focus groups in Malawi, people
with disabilities were invited to participate in other focus groups. Table 3 (below) provides a
consolidated snapshot of the numbers and types of informants for the RTE. Additional details on key
informants and the focus group discussions are provided in Annex 10.
20
Table 3 – Summary of key informants and focus group discussions
Regional and
global
Category
♂
♀
Total
Focus
group
discussions
held
UNICEF
10
15
25
0
Other interviewees
1
1
2
0
SUB-TOTAL
11
16
27
0
Malawi
♂
♀
Total
FGD
UNICEF
12
11
23
4
Other interviewees
22
16
38
1
Affected community
members
60
78
138
22
SUB-TOTAL
34
27
199
27
Mozambique
♂
♀
Total
FGD
UNICEF
24
14
38
0
Other interviewees
76
38
114
5
Affected community
members
43
80
123
20
SUB-TOTAL
100
52
275
25
Zimbabwe
♂
♀
Total
FGD
UNICEF
16
18
34
2
Other interviewees
39
26
65
3
Affected community
members
29
33
62
10
SUB-TOTAL
55
44
161
15
Overall
♂
♀
Total
FGD
UNICEF
62
58
120
6
Other interviewees
138
81
219
9
Affected community
members
143
207
350
52
GRAND TOTAL
343
346
689
67
30. During this RTE, the team applied UNICEF ethical guidelines for evaluations,
22
OECD/DAC
quality standards and UNEG ethical standards for evaluation, including those applicable to interviews
with children. Details are provided in Annex 11.
5. Constraints and limitations
31. The main challenges to conducting a quality evaluation were identified during the inception phase,
22
UNICEF, UNICEF Procedure for Ethical Standards in Research, Evaluation, Data Collection and Analysis, 2015.
21
and subsequent contingency planning helped to mitigate their effects. The main
challenges/limitations and how the team mitigated these are listed below:
32. Availability and quality of monitoring data. The nature of this sudden-onset emergency and
subsequent response – where capacity needed to be prioritized and the operating context was
changing rapidly – contributed to gaps in monitoring data. Apart from some UNICEF Zimbabwe
sectoral systems, humanitarian performance monitoring (HPM) systems
23
did not capture outcome
data. This was the case even when UNICEF’s implementing partners were generating outcome
information through their own internal outcome monitoring systems. Also, supply chain tracking
only covered a portion of the supply chain to targeted beneficiaries.
33. Interviews of key informants. The timing of the field visits of the evaluation team, more than three
months after the cyclone struck, coupled with staff turnover, meant that many individuals prioritized
for interviews were no longer in the country. Selected key informants who had previously been
involved were interviewed remotely or in person in their new duty stations so that the team had
various UNICEF and external perspectives for each phase of the response.
34. Perspectives from affected communities. Only one work week each was allocated for Malawi and
Zimbabwe, with two weeks allocated for Mozambique. Due to the short duration of the field visits,
the number of focus group discussions that could be disaggregated (e.g., focus on people with
disability) was limited. One result of this was that a relatively small number of people with disabilities
were interviewed, so much of the relevant data on how this population group was served by
interventions was derived from secondary sources, including staff from humanitarian agencies and
government and community leaders. Triangulation of results from focus group discussions and
interviews nevertheless enabled the evaluation team to gain a reasonable perspective on the response
vis-à-vis disaggregated groups.
35. Measuring performance. To measure the effectiveness of UNICEF’s performance, the TOR
expected this RTE to “…consider the HAC for each country, response plans, monitoring, adherence
to CCCs, etc.”. In addition to targets and benchmarks laid out in these documents, inter-agency
Humanitarian Response Plans, cluster strategies, government workplans and other documents could
be added. Together these represent a broad range of benchmarks and indicators, in addition to targets
that were not always coherent due to factors including rapidly evolving operating environments and
staff turnover. In line with good practice, the design of this RTE included participatory approaches
whenever feasible, and most of the outcome data were in the form of qualitative data collected during
focus group discussions with affected communities.
24
It was nevertheless evident that there were
differing expectations among UNICEF staff about the purpose, design and utility of this RTE.
36. RTE team coverage. Due to the relatively rapid mobilization of the RTE team, not all team
members were available to visit all countries due to prior commitments, although three team
members, including the Team Leader, were able to visit all countries. These gaps were mitigated by
designation of alternate focal points within the team.
37. Overall, there was a high level of engagement with the process by UNICEF staff and their partners.
Most viewed the evaluation as a useful exercise and this, along with the excellent logistics support
provided, greatly facilitated the team’s work to mitigate the constraints and help the evaluation team
develop a reasonable evidence base from which to draw concrete conclusions.
23
The HPM should capture both outcome and output level results as articulated in the CCC. See – Humanitarian Performance
Monitoring (PM) Toolkit.
24
See, for example, INTRAC, Real-Time Evaluation, 2017.
22
Response to evaluation questions
38. This section presents findings for each of the six Key Questions listed in the evaluation TOR. A brief
summary of findings for each KQ is followed by a narrative referencing subquestions in the
evaluation matrix. UNICEF’s responses in the three countries share common features, even though
the contexts in each country differed, notably in terms of the scale of the response. While high winds
inflicted considerable damage in Mozambique, especially in Sofala Province, the major impact in all
three countries was from flooding. Findings considering these contextual factors are presented below.
KQ 1: Relevance
KQ 1
How appropriate is UNICEF’s response strategy (present and planned) in reaching the most affected
populations?
39. This evaluation question looks at the extent to which UNICEF’s response was tailored to needs,
which assessment tools were used and the role played by communities in assessments. It also explores
whether the response was informed by preparedness and by coherence among sectors, among
partners and with the government.
Summary response to KQ 1
• UNICEF played a key role in inter-agency assessments, which helped ensure that UNICEF’s
emergency response strategies were aligned with priority needs in all three countries during the
emergency phase. Initial prioritization of the cholera response by UNICEF was appropriate and
helped avert a second disaster.
• UNICEF’s response strategies were designed to support the government’s lead role and there was
widespread appreciation by governments for UNICEF’s support.
• In common with other humanitarian agencies, UNICEF’s response strategy in most sectors
concentrated on displaced populations in camps in the most affected geographical areas. This
resulted in delayed response and incomplete coverage in Mozambique and Malawi.
• The influence affected communities had on response decision making was limited. Challenges in
accessing affected communities, procurement lead times, gaps in assessment data, and
preparedness and procurement planning all meant that some relief materials were distributed
when community needs were already shifting to recovery.
• Cash transfers did not fulfil their potential during UNICEF’s response.25
• Three months after Cyclone Idai, only UNICEF Malawi was found to have a comprehensive
multisectoral operational plan that took adequate account of early recovery needs. Plans and
guidance for a phased approach, including recovery planning, did exist at a sector/cluster level
in the other countries.
25
UNICEF Malawi was the only country that used a pre-existing cash transfer programme during the emergency phase of the
response, albeit after a few weeks’ delay. Non-use of this type of intervention in other countries was attributed primarily to
lack of cash programming in preparedness planning and resistance from some government authorities.
23
1.1 Alignment with the needs of affected populations
40. Feedback from affected communities during focus group discussions in the three countries
confirmed a clear alignment with priority humanitarian needs at the beginning of the response (i.e.,
during the first 4–5 weeks). Priority was given to cholera management by deploying WASH, health
and communication for development (C4D) capacities. Subsequent assessments indicated that
pockets of unmet needs emerged as the emergency unfolded across specific locations and sectors, as
well as cases of relief items whose quantities were insufficient to cover the needs. In Mozambique,
the relevance of the intervention was appropriate, but it was hindered by limited initial rapid and
detailed assessments at the district level.
26
41. UNICEF had clear criteria and approaches guiding its overall emergency operations, but with
some exceptions as described below. In Malawi and Zimbabwe, UNICEF had a reasonable
understanding of the situation of affected communities outside of sites for displaced people. This was
not the case in Mozambique, which raised questions about coverage that are addressed in more
detail under KQ 4.
42. Specific strategic gaps in UNICEF’s response included full integration of disability and gender
dimensions as foreseen in UNICEF response strategies;
27
absence of sufficient attention to
accountability to affected populations in preparedness plans; and lack of advocacy strategies for
addressing unmet needs in intervention areas where UNICEF did not have a lead role but which
nonetheless had a great impact on children, notably in the areas of shelter, livelihoods and
psychosocial support for adults. A view widely expressed by government key informants was that
international agencies, including UNICEF, should have allocated some of the resources for relief
supplies to repair and rehabilitate destroyed community infrastructure.
28
Apart from in Malawi, cash-
based transfers were not a significant component of any of UNICEF’s response strategies.
29
43. A challenge faced by UNICEF, particularly in Mozambique and Zimbabwe, was to make a
transition from emergency interventions to early recovery when there were large-scale
movements of people back to their communities or to alternative sites, including sites where
communities who were living in flood-prone areas would be permanently resettled.
30
Three months
after cyclone Idai, only UNICEF Malawi was found to have a comprehensive multisectoral
operational plan that adequately considered early recovery needs, although plans and guidance for a
phased approach, including early recovery, did exist at a sector/cluster level in the other countries.
31
1.2 Assessment tools
44. UNICEF played a key role in inter-agency assessments, which followed a similar process in
each of the three countries (see below).
26
WASH field assessment Chibabava 19, 20 June 2019; WASH field assessment Mandruzi 16 June 2019; WASH field visit Buzi
District 13-15 May; WASH field visit Muanza District 18 May.
27
Specific gaps are described in more detail under KQ4 and KQ6.
28
This was observed with WASH interventions in the form of repairs to water systems and construction of latrines in schools that
that had been destroyed. The perspective of the informants was also considering the delays experienced in procuring and
delivering relief supplies that resulted in reduced value-added for those communities that were already in recovery mode. Such
an approach is consistent with UNICEF’s global supply strategy on the humanitarian and development continuum.
29
This was attributed by the team to a combination of lack of familiarity with the use of cash-based systems in humanitarian
contexts and resistance by some of the governments. UNICEF Mozambique had just started a joint cash voucher programme
with WFP at the time of the RTE field visit.
30
In Mozambique, humanitarian agencies faced two additional challenges that impacted early recovery because the country was
struck by two successive cyclones within six weeks, along with the government’s programme of moving displaced people to
resettlement sites consulting with humanitarian agencies.
31
For example, UNICEF WASH sections and clusters in each country had a phased approach from emergency to recovery that
transitioned from communal infrastructures towards Community-Led Total Sanitation (CLTS) approaches.
24
Initial rapid assessments using aerial assessments or other remote assessment
methodologies due to difficulties in accessing affected areas (e.g., UNICEF Malawi
contributed to government assessments with their drones and technical support).
Participation in inter-agency multisectoral rapid assessments (MRA).
Cluster assessments that focused primarily at specific sectoral needs.
Post-disaster needs assessment (PDNA) led by governments around a month after
the disaster event, with technical assistance from World Bank, the European Union
and UN agencies.
45. A key challenge in the response, particularly in Mozambique and Malawi, where affected areas were
much larger, was acquiring a comprehensive understanding of needs. This was attributed to
several factors including a significant reliance on government data and the inability to establish a
system that was sufficiently dynamic for collecting, analysing and communicating assessment data for
population displacements and constantly changing needs. In Mozambique, humanitarian agencies
experienced challenges in accessing accurate census data and mitigating the risk of double-counting
populations that were constantly on the move. In Mozambique, UNICEF was one of the users of
IOM’s Displacement Tracking Matrix (DTM)
32
assessments for displaced populations.
46. UNICEF used RapidPro
33
as a data visualization tool in all three countries. There were successful
efforts in Mozambique in gaining government ownership of the tool, but government ownership
and use in other countries was limited. In Zimbabwe, UNICEF made use of RapidPro using data
from health facility personnel and over 300 village health workers to provide weekly reports on
numbers of malnourished children. UNICEF in Mozambique and Zimbabwe used C4D-led
activities, including U-Report, to regularly poll representatives in affected communities. Data
collection through U-Report was reported to be further amplified through national and international
partner members of the community engagement working group who were also using it. UNICEF
staff acknowledged that the U-Report sample was not sufficiently representative to be a valid and
reliable source of information on larger scale.
34
As the example from Mozambique shows (see Figure
10), U-Report was useful to get a snapshot of humanitarian needs, particularly where it was difficult
to access affected communities.
32
DTM Mozambique, Round 12, 31 December 2019.
33
See ONALABS’s Cyclone Idai Response Platform and ONA RapidPro.
34
See U-Report’s Mozambique cyclone response page.
25
Figure 10 – Results of U-Report poll in Mozambique conducted in March 2019
35
47. The lack of outcome monitoring in UNICEF systems made updating assessments and tracking
performance of humanitarian responses more difficult. This was true even though some UNICEF
partners were collecting outcome information through post-distribution/intervention monitoring.
36
1.3 Involvement of affected populations in the response
48. Focus group discussions and UNICEF key informant interviews in all three countries indicated that
affected communities had limited influence in decision making during the cyclone response.
Multisectoral rapid assessments carried out during the initial phase of the response included
consultations with members of affected communities but, with the exception of a reference to
community-based feedback systems in Malawi, accountability to affected populations was not
considered. Focus group discussions referred to only a few of UNICEF’s NGO partners as examples
of agencies which had consulted with communities.
37
These included Oxfam, which consulted
communities about integration challenges in resettlement sites in Mozambique, and Save the
Children, which along with Malawi Council for the Handicapped undertook participatory needs
assessments with communities in Malawi. Virtually none of the community focus group members
35
U-report opinion poll conducted a week after the cyclone (on 25 March) when many communities were inaccessible. 1,925
individuals out of 3,556 responded to six questions including: ‘What is your biggest need? What information would you most
like to receive? Any members of your household missing?’ Two U-report opinion polls were also conducted in Zimbabwe on
30 March to understand access to medication and health facilities for Cyclone Idai victims (12,042 responded out of 33,830
polled) with a follow-up poll on 29 April on Cyclone Idai-affected people (3,268 respondents out of 6,708 polled).
36
Examples reviewed include Action contre la Faim’s Mozambique Post-Distribution Monitoring Report (ACF, April-June 2019)
and Oxfam’s post-distribution report of Certeza in Beira_preliminary analysis results, 21 July 2019.
37
One of the conclusions of the RTE for the L3 response to Haiyan was that “UNICEF relies too heavily on its implementing partners to
ensure that the process of communication with affected communities works well.” Darcy, James et al., 2014, p. 43.
26
interviewed by RTE team members had been informed about the assistance that was going to arrive
or what the distribution criteria would be. Relief distributions were largely managed by chiefs and
there were allegations of sexual exploitation
38
and favouritism in some affected communities. Focus
group participants reported that, since quantities were rarely sufficient to cover all community
members, community leaders were often called upon to manage conflicts and facilitate a consensus
on how relief supplies should be allocated. On a more positive side, staff in schools spoke highly of
consultations with both UNICEF and partner staff. School staff members were not only aware of
planned assistance but were able to provide concrete examples of how UNICEF’s interventions had
been influenced by their advice.
49. In Mozambique
39
and Malawi, UNICEF actively supported inter-agency complaints and feedback
mechanisms that were adapted for use following the disaster.
40
While use of inter-agency CFM in this
context was consistent with good practice, the team found no evidence that UNICEF had internal
community complaints and feedback protocols in any of the three countries that could help
ensure and then process systematic feedback received from affected communities, whether via inter-
agency mechanisms or from other sources. From July 2019 onwards, UNICEF Mozambique, along
with UNHCR and IOM, supported Radio Mozambique programming in resettlement sites in Sofala;
this programming encouraged feedback from affected populations and disseminated early recovery
messages.
41
C4D also co-led the Community Engagement Working Group based in Beira.
1.4 Influence of emergency preparedness planning on the response
50. Prior to the cyclone, UNICEF had already been working with relevant government ministries and
departments at national and provincial/district levels. UNICEF’s longer-term resilience programming
with government counterparts included strengthening preparedness capacities, which supported
the government’s lead role in the response, something that received widespread praise from
government counterparts.
51. Emergency preparedness has been a key element of UNICEF Country Programme Documents
(CPD) developed in consultation with the government in each country. In Mozambique’s CPD for
2017–2020, emergency preparedness, together with resilience, climate change and disaster risk
reduction, was identified as a key cross-cutting area that supports the results of the programme.
UNICEF Malawi’s CPD incorporated building household resilience and improving early warning
mechanisms and data use to better predict and withstand shocks.
42
In Zimbabwe (CPD 2016–2020)
the number of sectors incorporating emergency preparedness (and related humanitarian and financial
capacities) was a progress indicator of the CPD’s results and resources framework, in recognition that
the country is prone to recurrent droughts and floods.
52. Preparedness varied by countries and sectors. Based on interviews with UNICEF supply staff and
an analysis of supply chains, contingency stocks were sufficient for approximately 10–20 per cent of
initial needs. None of the HCTs in the three countries had functioning pre-agreed multisector
assessment tools in place prior to the cyclone.
43
These were only developed after the disaster event.
Another gap in preparedness was the lack of clear intervention strategies, both by UNICEF and the
broader international humanitarian community, for communities living in hazard zones who were
subsequently permanently relocated to other locations.
53. WASH sections in all three countries demonstrated the highest levels of preparedness, with education
and child protection being less prepared, except in Zimbabwe, where the level of preparedness in
38
Pijoos, Iavan, ‘Cyclone Idai victims in Mozambique allegedly sexually exploited’, Sunday Times, 26 April 2019.
39
Linha Verde, managed by WFP.
40
UNICEF Malawi, Malawi Humanitarian Situation Report: Cyclone/flood Sitrep: March – June 2019.
41
UNICEF Mozambique, Mozambique Humanitarian Situation Report: Cyclones Idai and Kenneth Situation Report #13 –
August 2019.
42
The country office’s Annual Workplan indicates UNICEF actions to mitigate risk from emergencies and manage residual risk.
43
Governments had multi-sector disaster assessment templates, but these had to be adapted to meet the standards of UNICEF
and other international agencies.
27
the various sections was more consistent. Good examples of preparedness in WASH and other
sectors were attributed to a combination of the availability of pre-positioned stocks; standby PCAs
with experienced partners; the capacity of government counterparts to respond to quick-onset
disasters. In Mozambique and Malawi, one factor in the lack of preparedness in the child
protection section was the relative lack of trained and experienced staff. UNICEF Mozambique had
experienced education-in-emergencies staff based in Maputo but faced challenges identifying required
field-based surge staff, a gap that was only partially filled by short-term field missions.
54. Other sections were generally less prepared. With the exception of the cholera response, where the
health and C4D sections were active from the start in all three countries, this lack of preparedness
impacted the timeliness and coherence of the response. There was a heavy reliance on government
and external surge support to guide interventions and develop tools in real time and, challenged by
rapid turnover of many surge staff, managing and communicating relevant data within sections and
between sectors proved difficult. Specific areas that could be highlighted for each country are listed in
Table 4 below.
Table 4 – Contributing preparedness measures in each UNICEF country office
Mozambique
• In Mozambique, UNICEF’s warehouse in Beira containing contingency stocks was
destroyed by the cyclone and one of the first activities was a salvage operation, in which
more than 90 per cent of the stock was salvaged.44
• WASH had a particularly good level of preparedness, with the largest number of
contingency PCAs and long-term agreements in place; the ability to quickly get a surge team
sourced through standby partners; and relevant sectoral tools & standards in place.
• The government’s Strategic Plan for Education, developed with UNICEF support,
included preparedness at a national level. However, focus group discussions revealed a low
awareness of this in communities. Some stand-by PCAs were in place, but not all could be
activated due to lack of funding for supplies. With the exception of Save the Children, other
PCAs for education partners took an average of more than 80 days after the cyclone to be
signed.
• Since cholera was endemic in Mozambique, health preparedness was in place but
allocating enough resources and conducting capacity building in the midst of the emergency
proved challenging for UNICEF staff. There were relatively few contingency PCAs covering
multiple sectors which, moreover, proved difficult to activate.
• UNICEF adapted pre-existing partnerships with several local organizations to support
community engagement and community mobilization during the response.
Malawi
• Contingency planning and preparedness were embedded in UNICEF’s joint work
plan with the government and reflected the cyclical nature of emergencies in the country.45
• UNICEF signed umbrella PCAs with partners in October 2018.
• There was a contingency fund for disaster response ($500,000).
• Availability of UNICEF drones complemented government’s initial assessments.
• Identification of warehouse locations for specific supplies to be readily available (e.g.,
education used World Vision’s warehouse).
44
In contrast, there were no UNICEF contingency stocks in Capo Delgado when Cyclone Kenneth hit and, since the impact of
the cyclone in these areas was less than in areas affected by Idai, it meant that most communities were in recovery mode by
the time relief supplies arrived.
45
The country office’s 2019 Annual Workplan specified UNICEF actions to mitigate risk from emergencies and manage residual
risk.
28
• WASH: There was excellent cooperation with the Global WASH Cluster, and surge capacity
was immediately deployed. Prepositioned supplies were in place.
• Education: Stand-by agreements with government meant that a roster of 50 trained teachers
could be quickly deployed to reinforce capacities in affected areas.
• Nutrition preparedness for drought was adapted to floods. No external surge support was
required for nutrition.
• Shock-responsive social protection: UNICEF supported the scale-up of Malawi’s Social
Cash Transfer Programme for slow-onset food emergencies with technical support, and
these same systems and experiences were successfully applied to the flood response.
Zimbabwe
• There were contingency PCAs in place for WASH and child protection.
• Prepositioning of supplies for WASH, education and health.
Added value of anticipatory action
46
55. Anticipatory action is increasingly accepted by humanitarian actors as a key component of an
effective response. UNICEF did not yet have an institutional position of early action,
47
although
UNICEF has developed regional partnerships with, for example, national Red Cross and Red
Crescent Societies to strengthen both preparedness and early action.
48
56. Examples of anticipatory action were observed in Mozambique and Zimbabwe.
In Mozambique, contingency stocks were moved to Beira before the cyclone struck and UNICEF’s
WASH and health teams travelled to Sofala by road, although they subsequently had problems in
accessing Beira after the cyclone passed because the road was cut between Beira and Nhamatanda. In
Zimbabwe, contingency stocks were moved to the area the day before the cyclone hit so that
distributions could take place during the first week, even though road access had been cut off.
1.5 Coherence and integration of the response within UNICEF
57. The degree of multisectoral integration varied. There was generally good cooperation between
WASH, health, nutrition and C4D, with some gaps in terms of cooperation with and among other
sectors.
49
Although there were gaps in C4D preparedness, the CD4 team generally provided good
multisector support in all countries. The cholera response in each country provided a notable good
practice example of different UNICEF sections working together.
Cholera management – a coherent and effective response
A success story for UNICEF during the cyclone response was the containment of cholera, which
is endemic in all three countries and could have resulted in another serious disaster event.
Working with the World Health Organization and NGO partners, UNICEF WASH, health and
46
Anticipatory action, which is also sometimes referred to as ‘early action’ or ‘forecast-based financing’ is defined here as “…an
activity taking place between an early warning trigger or a high-probability forecast and the actual occurrence of the corresponding disaster in order to
mitigate or prevent the humanitarian impact of the anticipated disaster”. See Annex 7 for additional details.
47
Pichon, Florence, Anticipatory humanitarian action: what role for the CERF? Moving from rapid response to early action, Working Paper 551,
ODI, April 2019.
48
UNICEF Europe and Central Asia, Humanitarian Action for Children for 2019.
49
For example, UNICEF interviewees acknowledged there had gaps between WASH and education in Buzi district in
Mozambique.
29
C4D units in each country supported Ministry of Health preventive measures through hygiene
dissemination messages, water treatment and chemical distribution. UNICEF demonstrated
good preparedness, including the existence of contingency PCAs, prepositioned stocks and pre-
existing relationships that led to a smooth response. This was true even though response was
hindered by limited access to many affected areas during the initial stage of the response.
Figure 11 – Suspected cholera cases, Sofala Province (27 Mar – 2 Jun, 2019)50
In Mozambique, 800,000 vaccines were dispatched to supplement contingency stocks and the
WASH section was able to deploy a trained ‘shadow’ emergency team embedded in the unit. The
cholera response to Cyclone Kenneth benefited from lessons learned from the response to
Cyclone Idai. The WASH response included identification of cholera hotspots based on
epidemiological data in coordination with MoH partners, leading to daily targeting of WASH
interventions such as house-to-house C4D and household water treatment.
The WASH section in Mozambique provided a good practice model of preparedness for a
large-scale disaster response: staff in the section were routinely sent for disaster management
training; contingency stocks were regularly maintained; standby PCAs were in place; and there
was a high awareness among staff of protocols and the use of relevant tools. This meant that the
WASH section had staff with relevant emergency training and experience already in the country
who could quickly deploy, while at the same time calling upon external resources to reinforce
their capacity.
58. In Mozambique, the child protection section needed to work closely with other sections in Capo
Delgado to reach affected communities due to the access and security constraints caused by the
ongoing conflict, which hindered delivery of humanitarian assistance.
59. In Malawi and Zimbabwe, large segments of education budgets were allocated to WASH for
sanitation in schools. Integration between health and WASH was particularly important during this
50
Mozambique Ministry of Health, et al., Weekly Epidemiological Bulletin, Publication No. 5, May 27 – June 2, 2019.
30
response due to the relatively low rates of sanitation coverage prior to the disaster.
51
1.6 Complementarity of UNICEF’s activities with other humanitarian partners
60. Interviews with peer UN agencies, government counterparts and other partners confirmed that
UNICEF was viewed as a team player that actively participated in and supported inter-agency
initiatives.
52
Close collaboration with each Ministry of Health, national disaster management agencies
and WHO was a key element in the successful cholera management response in all three countries.
There was also close cooperation between UNICEF and WFP on nutrition, with WHO on
preparedness and monitoring of cholera and other communicable diseases and UNFPA and other
humanitarian agencies on integrated approaches to sexual and gender-based violence and PSEA.
53
Interviews with UNICEF staff and partners found that there was a good understanding of focal point
responsibilities among humanitarian partners. Examples of other collaborative activities specific to
each country are listed below.
Table 5 – Collaborative activities of UNICEF country offices
Mozambique
• UNICEF and WFP piloted a conditional cash voucher programme for 20,000 vulnerable
families for WASH and food items in July 2019.
• Collaboration with IOM for data collection and analysis for the Displacement Tracking
Matrix (DTM).
• Collaboration with UN-Habitat to rebuild school infrastructure.
• Helped the National Disasters Management Institute (INGC) to set up a data visualization
platform54 based on the ONALABS platform to reinforce their information management
and communication capacities.
Malawi
• UNICEF supported the Department of Disaster Management Affairs (DoDMA) with drone
technology55 for an initial survey of affected areas that were inaccessible by road.
• UNICEF supported the shelter response through a PCA with the Malawi Red Cross.
• UNICEF collaborated with WFP to mobilize resources for the response.
Zimbabwe
• Collaboration with IOM during data collection and analysis for the Displacement Tracking
Matrix (DTM).
• UNICEF’s child protection section collaborated with UNHCR and UNFPA to roll out
trainings on PSEA and protection mainstreaming for humanitarian actors.
1.7 Coherence of UNICEF’s response with government priorities
61. UNICEF’s cyclone response strategy was designed specifically to support the lead role of the
governments, and there was widespread appreciation by government officials during interviews of
51
As an example, only 43 per cent of people in Chimanimani district in Zimbabwe had access to latrines prior to the cyclone.
52
Examples are described elsewhere in this report and include UNICEF’s support for inter-agency PSEA initiatives and
UNICEF Zimbabwe’s lead technical advisory role during the HCT’s multi-sectoral assessment.
53
UNICEF Mozambique, Mozambique Humanitarian Situation Report No. 5, Cyclone Idai Situation Report 5 – 08-12 April 2019.
54
See https://cycloneidai.onalabs.org/. At the time the field mission took place, INGC was considering institutionalizing the
platform.
55
UNICEF, Drones in Malawi, UNICEF Factsheet, 2018.
31
the key role UNICEF played in supporting their response.
62. UNICEF’s response proved generally coherent with that of governments, with some
divergence around prioritization of intervention areas. For example, in Capo Delgado
(Mozambique), a key government criterion for interventions was priority needs, including in urban
areas. UNICEF, however, mainly focused its support on rural areas.
KQ 2: Achievement of results
KQ 2
To what extent has UNICEF achieved/is UNICEF achieving its intended results, and within the
planned timeframe? (consider HAC, response plans, adherence to CCCs, etc.)?
63. This question looks at the results of UNICEF’s response, including achievements against planned
results, the extent to which these planned results were updated as needed, the main contributing
factors and obstacles to achievement and the timeliness of the response. This question also explores
the role of innovation and whether there were any significant unintended consequences of
UNICEF’s interventions.
Summary response to KQ 2
• Performance varied by sector in the three countries, with good results achieved with WASH,
health and nutrition. Preparedness and effective partnerships contributed to the success of the
response, including the rapid and effective containment of cholera.
• Activation of L2 and L3 protocols resulted in effective and timely support by ESARO and
UNICEF headquarters, including an immediate allocation from UNICEF’s reserve funds to help
kick-start the response and early visits from senior managers to facilitate strategic decision
making.
• This quick-onset disaster highlighted the challenges of setting targets for the HAC due to gaps in
assessment data and a dynamic operating environment. Based on available evidence, UNICEF
met many of its targets, although performance varied by sector and country.
• Efficiency and effectiveness were influenced by the time needed for procurement, which resulted
in delayed distribution of relief items. In Mozambique and Malawi, the short duration and even
gaps in many (though not all) surge deployments also reduced response efficiency and
effectiveness.
• With the exception of staff in the supply section and some staff working in WASH, there was
little evidence that UNICEF staff were routinely considering cost when determining intervention
options.
• The governments can potentially mainstreamed some intervention elements and replicate them in
future responses. Notable among these were examples of the co-opting of the RealPro platform
by INGC in Mozambique, which should help address assessment gaps during future
emergencies.
2.1 Situation analysis and UNICEF targets
64. UNICEF country offices in all three countries were already responding to drought when the cyclones
struck. Whereas UNICEF Mozambique drafted a standalone HAC for cyclone response, the other
two countries revised their existing HACs in light of the cyclone. At the time the field mission for
this RTE took place, none of the HACs that were drafted in the aftermath of the cyclones had
yet been revised, although WASH and education sections had revised or were in the process of
32
revising their targets.
65. This quick-onset disaster highlighted the challenges of setting targets in the HAC due to
gaps in assessment data and a fluid operating environment. Achievements against HAC targets
reported
56
by UNICEF country offices are attached as Annex 4. It took time to gain a reasonable
understanding of needs, which were changing rapidly in any case, so the HAC targets were based on
projected rather than identified needs.
66. Situation reports and operational response plans provided a better benchmark for measuring
achievements; these indicated that UNICEF met many of its targets,
57
although performance
varied by sector and subsector. Table 6 (below) provides a summary of qualitative scores assigned
by sector specialists on the RTE team, using objectives described in country strategic plans (see Table
1, above). These objectives can be seen as indicators for measuring successful outcomes. Scoring was
based on an assessment of monitoring data from sources that included key informant interviews,
focus group discussions with affected communities and secondary data. The RTE team used a matrix
with CCC commitments and benchmarks to structure the analysis (see Annex 5). As mentioned under
limitations, UNICEF’s humanitarian performance monitoring (HPM) systems (with the exception of
those for some sectors in Zimbabwe) did not measure outcomes. The RTE thus needed to find
alternative ways of measuring outcomes and the major source for this was disaggregated focus group
discussions with affected communities that gauged the timeliness and quality of assistance. These
groups also helped the team assess UNICEF’s performance with respect to its commitments to
accountability to affected populations.
56
Based on available data. Data for some subsectoral activities in nutrition and education were not available.
57
Based on data in UNICEF sitreps and dashboards for each country. One reason for relatively low achievement in terms of
percentages of HAC targets achieved in Malawi was that targets were based on an annual plan.
33
Table 6 – Qualitative assessment of performance by sector and country against key objectives
of UNICEF’s response strategies
58
Sector
Performance
Supporting narrative
Mozambique
WASH
7
The WASH sector met or exceeded most targets except for
sanitation. Contributed to successful cholera management. 4W
coordination tools adapted to monitor WASH cluster
interventions. Innovative partnership with the private sector to
help ensure water purification products were available. Some delay
during the start-up phase in establishing the cluster.
Health
7
UNICEF leadership and coordination played a key role at all
levels. Key role in containing cholera. Use of Health Emergency
Week to ensure coverage with basic health services for affected
population. Expedited shipment of 800,000 cholera vaccines.
Some delay in achieving coverage of affected populations.
Nutrition
7
Good coverage for screening. UNICEF provided support for the
implementation of outreach activities, targeting hard-to-reach
communities and resettlement sites with the integrated mobile
brigades that provides a range of services including nutrition
screening and treatment, and infant and young child feeding
(IYCF) counselling, along with technical support. Some gaps in
national coordination.
C4D
6
Surpassed targets in terms of affected populations reached.
Partners gave positive feedback about C4D but felt it was not
sufficiently prioritized by UNICEF. Contributed to successful
management of cholera.
Education
5
Delayed response to Idai, resulting in relatively low achievement of
targets. Use of learning from Idai and improved pipeline
contributed to a more rapid response to cyclone Kenneth with
78% of targets achieved. Difficulties in identifying suitable
education in emergencies specialists. High rotation of field-based
staff with gaps in handover.
Child
protection
4
Appropriate focus on unaccompanied minors. Good cooperation
with other UN agencies. Delayed start-up. Low targets set relative
to identified needs. Support to people living with disability
provided later in response.
Malawi59
WASH
6
Met most targets. Immediate availability of flexible funding for
start-up and building resilience while responding. Installation of
cost-effective solar reticulation systems. Timeliness of the
response: almost all temporary solutions were completed on time.
Latrines completed in almost all locations, though sanitation
services in camps were judged insufficient.
58
Performance ratings are as follows: 8-10: high standard of performance; 6-7: good performance: 4-5: mixed performance: 2-3:
some shortfalls: 0-1: significant gaps.
59
WASH HAC figures for Malawi include both emergency and lean-season response.
34
Sector
Performance
Supporting narrative
Health
7
Adequate health services provided with fewer cholera cases than
during the previous year. Delayed response. Frequency of
UNICEF-supported mobile clinics was mixed. Spread of coughs
and bedbugs in centres for internally displaced people.
Nutrition
7
One of the first organizations to respond after the floods. Mobile
clinics screened under-fives. Delays in procurement of supplies.
C4D
5
Good cross-cutting approach that reached communities in
different forms. Delayed start-up and reached only a third of the
targeted population.
Education
5
Timely response that helped schools start operating with minor
delays. Provided tents, school kits and learning materials and
deployed 65 emergency voluntary teachers who had been trained
with UNICEF support before the cyclone. Only around half of
children targeted were reached. No data or targets for disability
and gaps in identification of gender-specific needs.
Child
protection
5
Achievement against targets show limited achievement at 13% of
targeted children receiving psychosocial support through safe
spaces. No specific support to people living with disability.
Provided training on preventing sexual abuse and exploitation
(PSEA) prior to deployment of surge staff.
Zimbabwe
WASH
7
Exceeded targets. WASH services included restoration of the main
water supply covering 25,000 people within three weeks,
supporting with emergency water trucking in the interim. Timely
response. Immediate needs for temporary latrines were met in
both schools and camps. Existing rural WASH information
management system effectively used as an assessment tool.
Health
8
Response was informed by needs, and targets were largely
achieved. Village health worker system functioned well. Timely and
good-quality surge.
Nutrition
7
Most targets were achieved or exceeded within the scheduled
timeframe. Activation of expanded protocols for management of
moderate acute malnutrition at the onset of the emergency. WHO
and UNICEF issued a statement recommending how IYCF
practices should be addressed.
C4D
8
Good coverage and consistent multisector support. No outcome
monitoring, although there was generally positive verbal feedback
from users of CD4 services.
Education
8
Exceeded targets set for the emergency, and the response was
timely and effective. Contingency stocks moved to impacted areas
two days before the disaster struck. Standby PCAs and processes
were efficient. Surge deployments were well planned with
reasonable duration and overlapping handover periods between
rotations. UNICEF covered two thirds of cluster targets for the
affected areas. Post-distribution monitoring results showed
distributions were efficient, with some duplications.
Child
7
Response was timely and achieved good coverage. Led a rapid
35
Sector
Performance
Supporting narrative
protection
assessment in the first week and strengthened capacity of social
workers and civil society organizations (CSOs). Quick at deploying
partners because UNICEF had PCA contracts with nine CSOs
which could be quickly activated. Proactive support from ESARO
ensured rapid identification and deployment of two surge staff
from the region. UNICEF played a key role in setting up the
PSEA network.
Source: KIIs, focus group discussions, secondary data and CCC matrix (Annex 5)
67. Additional findings on WASH, health, nutrition and education are given below. UNICEF’s other
focus area during the response, child protection, is covered under KQ3.
60
Mid-year situation reports
for each country provided the data for percentages and numbers reached.
WASH
68. WASH performance was generally good in all three countries, except for sanitation in Mozambique
and Zimbabwe and water in Malawi.
61
A major contributing factor to the overall success was the clear
intervention criteria set up nationally that applied a phased approach from emergency to recovery
from the onset of the emergency response. Good preparedness was a major contributing factor to an
effective response.
69. In Mozambique, WASH service coverage prior to the cyclone was already low and the disaster
aggravated the situation. WASH interventions in response to the cyclone were used as an opportunity
to raise standards. Initially, UNICEF focused emergency interventions on Beira to manage the
cholera outbreak. Despite delays in reaching some areas due to access challenges, WASH
achievements were significant, exceeding HAC targets and reaching some 1.5 million people with
water and more than 800,000 people with hygiene promotion in the Idai and Kenneth responses.
One of the main factors contributing to this success was effective partnerships with the government,
NGO implementing partners and private-sector contractors, not just during the response, but also in
joint preparedness prior to the disaster.
70. In Malawi, despite fluid population movements during the first few weeks following the floods,
virtually all temporary WASH interventions were completed within the planned timeframe. This
included 37,000 people reached with solar water reticulation systems out of a total of some 192,000
reached, or 39 per cent of the 496,000 people targeted in the HAC. Targets were exceeded for
sanitation, with a total of 56,041 people reportedly reached with sanitation facilities and 35,000 with
bathing units, exceeding the target set of 40,000 people.
71. In Zimbabwe, more than 865,000 people were provided access to safe water and personal hygiene,
more than twice the initial 325,000 targeted in the HAC. From an initial target of 500,000 people
receiving WASH-related information to prevent waterborne disease, more than 1.2 million people
were reached.
60
Additional details relevant to KQ2 are provided in Annexes 3, 4 and 5.
61
Numbers reached are based on data provided in mid-year reports by each UNICEF country office. Given the limited scope of
the RTE, it was not possible for the team to independently verify the figures. Based on perceptions of affected communities
visited and interviews with key informants, such as government counterparts with coordination roles, the team found no
reason to question reported figures.
36
Health and nutrition
72. UNICEF health and nutrition interventions used similar approaches in each country, supporting the
Ministries of Health with vaccinations, setting up cholera prevention and treatment centres, malaria
prevention and treatment, support to primary health care for internally displaced people in centres
and mobile clinics and recruitment of frontline health manpower. Nutrition work was integrated with
that of other sectors, notably WASH, health and child protection.
73. In Mozambique, health and nutrition interventions were somewhat slow to start but used the
opportunity of an Emergency Health Week in May 2019 to support MoH-led efforts that achieved
high coverage rates with vaccination, nutrition screening and basic health services for affected
communities in Sofala, Manica and Cabo Delgado provinces.
62
There was an initial focus on Sofala
Province and there were several weeks’ delay before severely affected areas in Manica received
attention. The impact of Cyclone Idai was seen as the main factor behind an outbreak of pellagra in
June, the first such outbreak in many years.
63
Out of a target of 500,000 children, 330,890 were
vaccinated against cholera and 673,641 against measles, with UNICEF support. Some 92 per cent of
the targeted 229,500 children under five received a consultation. Nutritional screening was done for
more than 770,000 under-fives, and 83 per cent were (of a targeted 100,000 children) were reached
with IYCF services and messages.
74. In Malawi, health and nutrition interventions were implemented according to the response plan.
Some 287,000 people were supported with access to health services, 68 per cent of those targeted,
and 42 per cent of a targeted 379,000 were immunized against measles. Targets for vitamin A
supplementation reached only around 60,000 out of a targeted 400,075. This was attributed to missed
opportunities to combine vitamin A supplementation with mobile health units. Just 38 per cent of a
targeted 58,421 children aged 6-59 months with severe acute malnutrition were admitted for
treatment.
75. In Zimbabwe, nutritional screening started at health facilities at the beginning of the cyclone
response. UNICEF also supported the MoH in establishing 12 temporary health facilities. EPI
campaigns and food distributions provided opportunities to widen assessments and vitamin A
supplementation in the affected areas. UNICEF and government key informants felt the opportunity
to conduct nutrition screening during the cholera vaccination campaign
64
was missed; this happened
several weeks later during the oral cholera vaccination (OCV) 2 and EPI and measles catch-up
campaign. Out of a targeted 73,640 children, 61,162 (83 per cent) were vaccinated against measles
with UNICEF support. Targets for vitamin A supplementation were exceeded by over 150 per cent,
with 207,451 out of a targeted 136,000 children reached. In addition, 29 per cent of a targeted 27,000
children aged 6-59 months with severe acute malnutrition were admitted for treatment. Numbers of
admitted children were used to estimate cases of malnutrition.
Education
76. Schools, churches and other public buildings provided temporary refuges in all three countries during
and after the cyclones, causing disruption of classes, even before internally displaced people whose
houses were destroyed or damaged were moved to temporary accommodation centres. A priority for
62
Mozambique Ministry of Health and the Global Health Cluster, Health Cluster Bulletin #7, May 2019.
63
UNICEF, ‘Mozambique: Children living in storm-affected areas face worsening food insecurity and nutrition crisis six months
after Cyclone Idai’, press release, 14 September 2019.
64
There were only 10 ward coordinators for nutrition projects (UNICEF supported), whereas health efforts relied on 300 village
health workers (volunteers) in Chimanimani and 500 in Chipinge.
37
UNICEF’s response was to create conditions for children to go back to school as soon as possible.
Depending on the area, this typically occurred 4-6 weeks after the cyclone.
65
77. In Mozambique, UNICEF’s initial focus was on assisting displaced people as they moved from
accommodation centres in Sofala to camps. UNICEF work included setting up temporary learning
spaces to accommodate children from nearby communities where schools had been damaged.
Another factor influencing education implementation was the priority given to health and WASH
when planning the supply pipeline. Challenges listed in Table 6 (above) contributed to difficulties in
reaching HAC targets. Out of a targeted 380,000 children, by mid-2019 only 16 per cent had been
supported by UNICEF to access education. Similarly, UNICEF reached only 7 per cent of the 76,000
children targeted to access play-based learning.
78. In Malawi, UNICEF focused its response on providing tents, school reconstruction, materials,
school feeding and support to the cluster system at the district level. The response was generally
viewed as insufficiently timely by focus group discussions and key informants, with the response
coming two weeks after the disaster in southern Malawi. In Chikwawa District, focus group
discussions said that UNICEF took a month to arrive. By May 2019, 66 per cent of the 91,000 school
age children targeted to access quality education services had done so with UNICEF’s support.
66
79. UNICEF Zimbabwe reached only 31 per cent of its initial HAC target of 225,000 school-aged
children accessing formal or non-formal basic education. The government’s March cyclone response
plan for education set much lower targets and considered only formal education in line with the
Rapid Impact and Needs Assessment. The subsequent Rapid Joint Education Needs Assessment in
May 2019 found that UNICEF-supported activities had covered 78 per cent of needs. UNICEF
subsequently revised its HAC to target 120,000 children.
Communication for Development (C4D)
80. C4D played a key supporting role during the response, with UNICEF adapting C4D initiatives
through strengthening its collaboration with existing partners
67
and also establishing new
partnerships. Key messages focused on cholera and malaria prevention, breastfeeding, vaccination,
HIV and AIDs, child protection, sexual and gender-based violence, PSEA and improved WASH
practices. In Mozambique and Zimbabwe, UNICEF used U-Report to conduct assessments and
convey key messages. In common with other sectors, there was little outcome monitoring of C4D
activities until the time of the field visit by the RTE team in July. UNICEF Mozambique did
eventually implement third-party outcome monitoring after the RTE team visit; household surveys
there yielded generally positive attitudes, practices and beliefs about key topic areas and some
familiarity with key topic areas.
68
Value for money
81. UNICEF’s risk management principles include accepting risk when benefits outweigh costs while
65
Zimbabwe was the least affected since the cyclone struck just before the school holidays, so children were able to return to
school on schedule.
67
In Mozambique, for example, partners included Cruz Vermelha de Moçambique (Red Cross of Mozambique), Instituto de
Comunicação Social (Institute of Social Communication), Rádio Moçambique and Programa Inter-Religioso Contra a Malária
(Inter-Religious Programme Against Malaria).
68
Forcier (2019) Third Party Monitoring of UNICEF’s Emergency Response Communication Activity Platforms: Baseline
Report, September 2019. The report cautioned that this was actually not a ‘pure’ baseline study due to the timing. Since C4D
activities had already been taking place for several months, it was assumed that knowledge and practices had been changing
over the intervening months.
38
anticipating and managing risk by contingency planning and mitigating identified risks.
69
This includes
contingency stockpiling of emergency supplies, which was a key feature of the cyclone response.
UNICEF’s preparedness to initiate action and emphasize the importance of thematic funding as a
flexible and efficient mechanism with fewer transaction costs also enhanced value for money. The
response to this quick-onset disaster was an example of the three country offices needing to ensure
that assistance was timely while striking a balance between reaching large numbers of people who
were easier to access and smaller numbers of those most in need, who were more difficult and
costlier to reach.
82. Based on interviews with staff from governments, implementing partners and cluster members it was
clear that there was a shared perception that UNICEF had added value to the response. At the same
time, with the exception of staff in Supply Division and some in WASH,
70
there was little evidence
that UNICEF staff routinely considered cost when choosing intervention options to balance
cost effectiveness and quality (including timeliness).
71
This was attributed partly to UNICEF’s
financial systems, which are not as conducive to extracting data for VFM analysis (such as activity-
based costing) as can be done by some other humanitarian agencies.
72
2.2 Factors that contributed to and inhibited the response
83. Factors that significantly influenced UNICEF’s response (as identified by key informants based on
strategic objectives in Table 1, above) are listed below in Table 7. Factors common to all countries
that facilitated the response are followed by those which are country-specific. Following this is a of
factors that significantly inhibited an effective response.
69
UNICEF, Core Commitments for Children in Humanitarian Action, 2010, p.13.
70
WASH staff in Malawi were able to provide the team with an analysis of water systems that informed decision making. For solar
reticulation systems the per capita cost was an estimated $,20 per person while per capita cost for handpumps was $30 per
person. Solar systems were also more cost effective in terms of operation and maintenance.
71
Findings were consistent with those in an evaluation synthesis that found inconsistent UNICEF approaches to cost effectiveness.
A common finding was that during an emergency response there was a tendency by UNICEF staff “…to place the principle
of ‘humanity’ above other principles, which is frequently (and incorrectly) interpreted as achieving humanitarian access and
assistance ‘at all costs’…”, from UNICEF, Evaluation of the Coverage and Quality of the UNICEF Humanitarian Response in Complex
Humanitarian Emergencies, 2019, p. 11.
72
Baker, Jock and Mark Salway, ‘Development of a proposal for a methodology to cost inter-agency humanitarian response plans’,
IASC, 2016.
39
Table 7 – Factors contributing to success / factors that inhibited the response
Factors contributing to success common to the three countries
• Significant amount of ‘no regrets’ funding from the UNICEF headquarters reserve, along
with some reprogrammed donor funds, meant that initial funding was not really an issue so
the response could kick in rapidly.
• Prioritization of the response by ESARO, which translated into timely and effective
technical and operational support in the form of regular missions and the mobilization of
surge staff from the region.
• Leaders of sections had emergency experience. There were emergency focal points in each
section.
• Good pre-existing relationships with government counterparts.
• Good levels of preparedness, including joint preparedness with government counterparts,
and contingency stocks.
• Declaration of L2 and L3 emergencies translated into higher visibility and prioritization by
UNICEF organizationally.
• UNICEF’s key role in cholera management resulted in unsolicited offers of funding from
donors. Donors showed flexibility in re-programming funding.
• Support from the regional office and Supply Division in Copenhagen for prepositioning
supplies prior to the emergency and emergency procurement during the response.
• Support from the regional office for surge staff. Supply Division deployed surge on 6-week
rotations supported by good-quality handover processes.
Factors contributing to success – Mozambique
• Relatively strong capacity of the national disaster agency, INGC. Rapid deployment of
government officials with disaster management experience to lead and coordinate the
response.
• Dedicated national Emergency Coordinator in the UNICEF country office.
Factors contributing to success – Malawi
• Effective integration of resilience into longer-term programming.73
Factors contributing to success – Zimbabwe
• Relatively small-scale and limited geographical area of the response.
• Dedicated national Emergency Coordinator in the UNICEF country office.
Factors inhibiting the response common to the three countries
• Preparedness had focused on drought, floods and periodic cholera outbreaks, not large-scale
cyclones.
73
Further details of UNICEF Malawi’s approach to resilience, notably cash-based social protection, community-based nutrition,
education and communication programmes can be found in Centre for Development Management, Qualitative Impact
Assessment of the Joint UN Resilience Project in Malawi: Implemented in Phalombe District in Southern Malawi – 2014 to 2016, 2016.
40
• Initial multisectoral needs assessments and initial responses were difficult in all three
countries due to lack of access, technical limitations and lack of expertise of the government.
Clear assessment was also hindered by having several competing assessments from partners
with different data and coverage.
• Gender and disability not systematically considered during assessments and interventions.
• Challenges in identifying and following up on cases of sexual and gender-based violence;
survivors also had insufficient trust in the system to come forward.74
• Lack of internal UNICEF systems around accountability to affected populations to promote
information sharing, participation and follow-up on feedback and complaints.
• Little previous experience of local governments in leading and coordinating a major
humanitarian response.
• Challenges with submitting supply plans in a timely way given uncertainties about available
funding and projected needs. Competition over airlifting relief materials.
• Limited experience within UNICEF of working with the private sector in emergency and
recovery operations.
• Lack of advocacy strategies in UNICEF’s response plans to promote better coverage of
sectors where UNICEF did not have a lead role but which nevertheless had a significant
impact on children, such as food security and shelter.
• While UNICEF sections had emergency focal points, their role was often focused on
information management; operational capacity to prepare for and respond to disasters varied
significantly between sections.
Inhibiting factors – Mozambique
• The duration of many internal and external surge appointments was only 2-3 weeks,75 so
very limited. This resulted in a high turnover of staff that was widely viewed by UNICEF
and external interviewees as disruptive for both the UNICEF programme and cluster
coordination.
• Limited use by some sections of existing in-country emergency capacity for response and
coordination (experienced emergency staff in Maputo rather than Beira) and heavy reliance
on external surge.
• Finding Portuguese-speaking surge was a challenge, especially at the strategic level. For
technical specialties, language was not considered as much of an issue and indeed the
language requirement was seen by some UNICEF and partner staff as inefficient.
• Government reservations regarding use of cash for an emergency response.
• Lack of familiarity of some section heads with IASC scale-up structures and protocols.
• UNICEF’s protracted delays in finalizing PCAs,76 particularly in the education and child
74
The WFP-managed Linha Verde call centre went live only at the end of May and, at the time the field visit took place during
early July, had only received a small number of PSEA-related complaints. Interviewees from UNICEF and other
humanitarian agencies in Mozambique felt many instances were not being reported.
75
An exception was the Supply Division, which sent surge staff on 6-week assignments, supported by systematic handover
processes.
76
Partners attributed these delays in large part to the rapid turnover of UNICEF focal points that resulted in constant restarting of
PCA negotiations. Education received only a little over 50 per cent of education kits over a 3-month PCA. According to
partners, apart from delays in delivering assistance, this resulted in cost increases of materials and partners incurring unforeseen
costs while waiting for the PCA to be activated.
41
protection sectors, which resulted in delays in procurement and distribution of relief items.
• Communication gaps between the UNICEF country office and the hub.77
• In Cabo Delgado, UNICEF had no permanent presence prior to the cyclone and was
challenged by difficult physical access and security restrictions.78
• Limitations posed by the government in terms of types of supplies (e.g., only Certeza water
treatment product could be distributed) contributed to a less than optimal response.
• Health promoters in affected areas had lost everything during the cyclone, including basic
health kits, which had to be replenished.
Inhibiting factors – Malawi
• Coordination and management gaps between the UNICEF country office and the hub.
• Limited duration of internal and external surge of 2-3 weeks resulted in a high turnover of
staff that was widely viewed by UNICEF and external interviewees as disruptive for both the
UNICEF programme and cluster coordination.
Inhibiting factors – Zimbabwe
• Challenges with coordination at a national government level.
• The economic situation was challenging due to sanctions and other factors. Government
authorities were not receiving regular salaries and had trouble obtaining fuel during the
response.
Surge staff supporting the response
84. As shown in Figure 12 below, there were a total of 168 individuals deployed from outside the country
offices under various surge mechanisms. Of these, 110 were deployed to Mozambique, 18 to
Malawi and 40 to Zimbabwe. Half of the surge deployed to Mozambique came through the
humanitarian surge deployment (HSD) mechanism and UNICEF staff deployed from other offices;
the primary source of external surge staff for Malawi and Zimbabwe was international consultants.
Figure 12 – Source of surge staff deployed to each country office
79
77
Interviewees partially attributed this to the emphasis given to ‘feeding’ the outside machine (e.g., the need to contribute
information for external communications generated by PFP for NatComs and the private sector) at the price of insufficient
time for internal communication.
78
The first tarpaulin tents were distributed three months after the sudden onset emergency due to access and security issues, at a
time when most communities were already in recovery mode.
79
Data from UNICEF HR sections in each country as of July 2019. Surge mechanisms: IC = international consultant, SBP =
standby partner, RRT = rapid response team, FST = field support team, HSD = humanitarian surge deployment, ERT =
emergency response team, RRM = rapid response mechanism, RRR = rapid response roster.
42
85. Key informants from UNICEF, partners and government authorities widely flagged the rapid
turnover of UNICEF surge staff
80
in Malawi and Mozambique as a negative influence on the
efficiency and effectiveness of the response. This was not the case with standby partner deployments,
whose deployment averaged 3-6 months,
81
compared to 2-6 weeks for UNICEF-sourced surge for
this response. Reliance on the humanitarian surge deployment (HSD) mechanism was one of the
main drivers of rapid turnover, since UNICEF does not have minimum requirements for deployment
duration for staff deployed on surge. In Zimbabwe, deployment lengths for surge staff, and their
overall value-added, were perceived positively by UNICEF Zimbabwe staff and external key
informants.
86. Short rotations were aggravated by gaps between deployments and inadequate handovers. They were
seen as an obstacle to developing working relationships with government counterparts and other
humanitarian agencies and to building UNICEF teams in newly formed hubs. Short deployments also
fostered various inefficiencies that resulted from staff working in an unfamiliar operating
environment.
82
There were also significant transaction costs for UNICEF’s HR staff and UNICEF
hub managers, who had to ensure staff received orientations and had somewhere to stay and work,
sometimes with little or no advance notice of their arrival. Short deployment lengths were also viewed
as a contributing factor to the lack of awareness about UNICEF’s role and plans for assistance within
affected communities.
83
Previous evaluations indicate that this has been a constant challenge that
UNICEF has not yet adequately addressed but which has had a significant negative impact on
efficiency and effectiveness, notably from an AAP perspective.
84
Interviews with UNICEF staff
80
This was the case for both UNICEF staff redeployed by the country office on temporary assignments and sourced from
regional and global rosters. The data for surge presented here do not include short-term field missions by staff from the
country office, ESARO or UNICEF headquarters.
81
Standby Partnership, After Action Review: Tropical Cyclone Idai Response, 2019.
82
This was also true for UNICEF national staff deployed as surge staff in Mozambique, because many were unfamiliar with the
operating context in areas affected by Idai and Kenneth, didn’t speak the local languages, and had no local networks.
83
In one example of this, leaders in a village in Manica Province that had been almost completely submerged during the flooding
said that they had seen UNICEF cars but didn’t know any of the staff.
84
See, for example, Bhattacharjee, A. et al., Independent Review of UNICEF’s Operational Response to the January 2010 Earthquake in Haiti,
43
indicated that key constraining factors to longer deployments included difficult living and working
conditions during the initial phase of the response along with the lack of incentives and associated
protocols that would encourage longer deployments.
87. An analysis of data provided by UNICEF found an average deployment length of 15 days in Malawi,
30 days in Mozambique and 43 days in Zimbabwe (Figure 13). As mentioned above, surge
deployments lengths were viewed as appropriate in Zimbabwe, but not in Mozambique or
Malawi.
85
When key informants were asked about the duration of UNICEF surge deployments in
Mozambique, the most common response was ‘two to three weeks’ which is consistent with a field
deployment for 15-30 days that, for staff from outside the country, included travel, orientation and
administrative formalities. Some short-term deployments could be justified, notably for staff tasked
with such time-limited tasks as installing communications equipment or, as in the case of
Mozambique, where most staff deployed on surge were replaced by staff with temporary assignment
contracts within two weeks.
Figure 13 – Duration of surge deployments by country and functional area (days)
86
2.3 Timeliness of the response
88. UNICEF was one of the international humanitarian agencies that began mobilizing as soon as the
harmful potential impact of Cyclone Idai became evident. Government interviewees in all three
countries stated that UNICEF was among the first international agencies to arrive and provide
assistance. The main obstacle all country offices faced in mobilizing their respective responses was
the lack of physical access to the most-affected areas because roads and bridges had been washed
away. The speed of response varied by sector, with UNICEF’s WASH section being particularly rapid
in all countries. Surge staff were rapidly mobilized from within the countries, from the regions and
from UNICEF’s global surge roster (including standby partners).
89. In Mozambique alone, 49 surge staff were deployed within the first three weeks.
87
L2 and L3
protocols were activated and the ESARO Regional Director cut short a mission to the Democratic
Republic of the Congo and arrived within days of the disaster to support the scale-up. The EMOPS
Director and Deputy Regional Director followed soon after. A key catalyst to the timely scale-up of
the response was the rapid funds allocation on a ‘no regrets’ basis from the UNICEF reserve which,
for Mozambique alone, amounted to $8 million.
90. WASH and health, supported by C4D, were prioritized during the response. Education and child
protection were somewhat slower in responding due to a combination of lower priority given to these
sectors, lack of experienced emergency staff (including surge) and gaps in preparedness such as lack
UNICEF, 2010.
85
Deployment lengths were longer for surge deployed by Supply Division (6 weeks) and standby partners.
86
Source: UNICEF HR units in each country.
87
UNICEF HR data.
44
of standby PCAs and streamlined contracting processes. PCAs for child protection and education
were signed an average of 77 days and 87 days, respectively, after the cyclone; many of the associated
activities for these sectors were only just getting under way in Mozambique during the RTE team’s
field visit at the beginning of July.
91. In Malawi and Zimbabwe, the education and child protection sections mobilized more rapidly they
did in Mozambique, due to inhibiting factors for Mozambique listed above. In Zimbabwe, case
management officers from unaffected districts who were supported by UNICEF to mainstream child
protection issues arrived within days. UNICEF surge arrived one week after the cyclone to help set
up a psychosocial support committee and child support committee which were operational within
two weeks. Similarly, support from UNICEF and its partners ensured that schools were able to start
immediately after the holidays had finished, which was just over a month after the cyclone.
92. A significant amount of UNICEF’s budget was allocated to procurement, amounting to a total of
around $17 million each for Mozambique and Zimbabwe and some $13 million for Malawi.
88
After in-country contingency stocks had been depleted, UNICEF struggled in all three countries to
fill the international supply pipelines in a timely way; this resulted in relief items being delivered to
UNICEF warehouses or handed over to UNICEF implementing partners an average of eight to
eleven weeks after the cyclone (Figure 14).
Figure 14 – International supply pipeline timelines (Mar 17 – Jul 31, 2019)
89
93. UNICEF supply tracking systems did not provide data on when relief goods actually reach affected
communities, but the result was that relief supplies were eventually being distributed to many affected
communities that were already in recovery mode, when emergency assistance was no longer a priority
need.
94. Timeliness was affected by a combination of external and internal factors. External influences
included lack of complete assessment data from government and other sources (Mozambique and
Malawi), difficulties in accessing affected areas, limited cargo space on humanitarian flights
(Mozambique) and delays in clearing international shipments (Zimbabwe) and limited procurement
from regional suppliers, notably South Africa. UNICEF-specific issues included difficulties providing
supply plans in a changing and uncertain operating environment. Deliveries were prioritized by sector
by each country office so that, for example, the average time for international deliveries of oral
cholera vaccines to Mozambique using global freight forwarders was only 12 days, whereas bed nets
88
Source: UNICEF Supply Division.
89
Source: UNICEF Supply Division. Acronyms: SO = sales order, PO = purchase order, GR = goods receipt, IP =
implementing partner, SD WH = Supply Division-controlled warehouses. VISION data for Zimbabwe was incomplete so is
only an estimate.
45
took more than 5 weeks to be delivered to implementing partners or UNICEF warehouses in that
country.
95. Local procurement was an average of 20 per cent quicker than international supply procurement in
Mozambique. It was slower in Zimbabwe and Malawi due to longer processing and delivery times
(Figure 15).
Figure 15 – Local procurement supply timelines (Mar 17 – Jul 31, 2019)
90
96. At a country level, UNICEF staff reported challenges with using the data-heavy SAP tracking
software in areas with no or poor Internet connectivity, lack of a user-friendly interface for UNICEF
programme staff to track relief commodities and the limited number of long-term agreements (LTAs)
with South African suppliers that could have reduced reliance on air transport shipping of
commodities through Beira.
2.4 Innovation in assessments and during the response
97. UNICEF innovations during the response to the cyclones can be divided into four categories: use of
data visualization using the ONALABS platform,
91
use of Kobo IT tools, SMS-based community
engagement and, in Malawi, piloting of early warning systems and use of drone technology to
support government-led assessments in affected areas. All UNICEF country offices were using these
tools prior to the cyclones for their longer-term programming. The primary innovations during the
cyclone response were making necessary modifications to support emergency response.
98. Given the challenges (described above) with understanding coverage of assistance and priority needs,
Rapid-Pro was a useful tool. In Mozambique, INGC (the national disaster agency) was appreciative
of UNICEF’s help with its own communications and was exploring with UNICEF whether the tool
could be institutionalized to support future emergency response. While improving data visualization
in support of the emergency response, Mozambique and Malawi faced continuous problems with
harmonizing and cleaning input data from different sources.
99. UNICEF country offices in Mozambique and Zimbabwe both made use of U-Report to collect
relevant information from communities, disseminate key messages in communities and support
90
Source: UNICEF Supply Division. International procurement represented 60 per cent of total procurement in Zimbabwe, 81
per cent in Mozambique and 93 per cent in Malawi.
91
https://cycloneidai.onalabs.org/.
46
health campaign mobilization. Since a U-Report sample is not based on statistical sampling methods,
the resulting data was viewed as indicative rather than an accurate depiction of community
perspectives.
100. UNICEF Malawi drew upon its ongoing work with a Brazilian university (Pernambuco) in predicting
and preventing emergencies through use of bio-epidemiological platforms to predict flood water
levels and diseases. UNICEF Malawi also deployed its drone, along with a technical specialist, to
support the government’s initial assessments. This support was welcomed, although interviewees with
both government and UNICEF staff acknowledged that the mapping of disaster impacts could have
been done in a more coordinated way to improve utility.
92
In WASH, the engagement of the Malawi
WASH team with the University of Strathclyde and the Government of Scotland allowed for the use
of mWater, an innovative geo-referenced database, for borehole identification.
93
2.5 Unintended consequences of the humanitarian assistance
101. The main positive unintended results of the response which were not part of the initial response
strategies ended up as joint initiatives that had the potential to be mainstreamed by the
government and be replicated in future responses. Notable were examples from Mozambique of the
co-opting of the ONALABS platform by INGC
94
and the joint cash voucher programme with the
government and WFP.
95
Another positive result observed in all three countries was that the response
provided opportunities for UNICEF to increase its longer-term programming in affected
areas. In Mozambique, this included working in districts where UNICEF had not been working
prior to the cyclone.
102. The negative consequences of short duration surge deployments are described above. Also, high
standards of preparedness for floods and drought were observed to only partially translate into an
effective response for a large-scale quick-onset emergency. This was less of an issue in Malawi,
which had incorporated lessons from the 2015 flood response, and in Zimbabwe, where the scale of
the emergency was relatively small and more manageable. All countries were nevertheless struggling
with balancing assistance in camps/sites with outstanding needs in affected communities, notably
those hosting large numbers of displaced people.
KQ 3: Coverage
KQ 3
To what extent was the affected population, including persons with disability, adequately identified,
targeted and reached by UNICEF and its partners?
103. This question looks at the extent to which UNICEF was able to assist hard-to-reach communities
and vulnerable groups and the extent to which vulnerable groups were able to participate in response
implementation and monitoring. This question also assesses the effectiveness of child protection
systems in reaching vulnerable groups and individuals.
Summary response to KQ 3
• Assessments included identifying vulnerable groups during the response, principally via collection
92
See, for example, Hein, D. et al., ‘An Integrated Rapid Mapping System for Disaster Management’, The International Archives for
the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XLII–1/W1, 2017.
93
Van den Homberg and Susha, I., Characterizing Data Ecosystems to Support Official Statistics with Open Mapping Data for
Reporting on Sustainable Development Goals, 2018.
94
UNICEF Mozambique, Mozambique Humanitarian Situation Report: Cyclone Idai Situation Report #5 – 08 – 12 April 2019.
95
There was also a similar joint initiative following the 2017 hurricanes in the eastern Caribbean. See Avilar, Carlos, Process Review
of the UNICEF-WFP Joint Emergency Cash Transfer (JECT) Programme in Dominica: Final Report, UNICEF Office for the Eastern
Caribbean Area, 2019.
47
Summary response to KQ 3
of disaggregated data and compilation of lists by UNICEF’s partners, although humanitarian
agencies in Mozambique and Malawi faced challenges with gaps in assessment data during the
response in terms of prioritizing assistance to vulnerable groups.
• Assistance targeting was largely controlled by community leaders, which resulted in allegations of
favouritism and sexual exploitation. UNICEF support to inter-agency complaints and feedback
mechanisms in Mozambique and Malawi was consistent with good practice, although none of
the three UNICEF country offices had internal protocols for following up on complaints or
feedback from communities affected by disasters, whether coming from inter-agency CFMs or via
alternative channels.
• UNICEF emergency interventions initially focused on areas that appeared to have been hit the
hardest and were in the most need of humanitarian assistance. This resulted in uneven coverage
of needs in Mozambique and Malawi.
• Apart from examples from the WASH sector, there was little evidence of targeting people with
disability.
• In Zimbabwe, the response and coordination for child protection was relatively quick and had
good coverage, but the child protection response struggled to gain momentum in the other two
countries. Zimbabwe faced a particular challenge with PSEA allegations arising from the military’s
involvement in relief.
3.1 Reaching vulnerable groups
104. Assessments included identifying vulnerable groups during the response, principally by collecting
disaggregated data and compilation of lists by UNICEF’s partners (working in child protection and
other sectors). Categories assessed included people with disability, unaccompanied minors and
women and girls assessed to be at risk.
105. The most significant challenges in identifying and reaching vulnerable groups were experienced in
Mozambique due to the scale of the emergency in terms of numbers and area affected, and also
because of the continuous movements of displaced communities. As with other realms of
intervention, there was more attention paid to Sofala Province – notably locales readily accessed from
Beira, with relatively less attention paid to identifying and assisting vulnerable groups in other
affected areas.
Geographic coverage
106. UNICEF emergency interventions initially focused on areas that appeared to have been hit
the hardest and were in the most need of humanitarian assistance. UNICEF had ongoing
programmes in all areas impacted by the cyclones, apart from Sofala and Manica provinces in
Mozambique. Where UNICEF had no offices prior to the cyclone, they quickly established
operational hubs in the vicinity of affected areas in each country.
107. In Mozambique, focusing assistance on Beira in Sofala Province was justified by the potential for a
cholera epidemic there. After the first week, health and WASH extended their geographic coverage to
focus on both camps and people outside of camps; they prioritize large urban and government health
facilities and disrupted water supply systems, covering all affected provinces (except for Buzi and
Chibabava districts in Sofala Province and Macate District in Manica Province). In Cabo Delgado
Province, coverage was limited primarily because of security constraints. In Cabo Delgado’s
provincial capital Pemba, assistance was provided to populations living in their places of origin and
those people staying in two transit centres considered to be amongst the hardest hit by clusters,
INGC and OCHA. Several other areas (e.g., Macomia District, where a reported 40,000 people were
severely affected by lack of water and power as well as pre-existing security issues) had not yet been
reached by humanitarian assistance as of early July.
48
108. In Malawi, UNICEF’s response covered the six hardest hit districts, i.e. Chikwawa, Mulanje, Nsanje,
Phalombe, Zomba, Machinga. The emergency response was initially focused on southern districts,
which had the highest concentration of displaced communities and where UNICEF had been
running long-term programmes addressing needs related to the area’s vulnerability to the effects of
climate change. As in Mozambique, UNICEF tended to prioritize displaced communities living in
camps. This meant that Machinga and Zomba districts in eastern Malawi, which had smaller
numbers of displaced communities yet larger proportions of affected households, only started
receiving assistance 4-5 weeks after the disaster.
109. In Zimbabwe, only two districts (Chimanimani and Chipinge) were significantly impacted by
Cyclone Idai, and UNICEF assistance was focused on vulnerable communities there. After the
emergency phase ended in June, UNICEF assistance mainly focused on some 60,000 members of the
affected communities who remained in camps for the displaced.
Disability inclusion
110. Focus group discussions in each country mentioned some examples of interventions targeted at
vulnerable groups, including psychosocial support, adapted latrines in schools and camps and non-
food item distributions that prioritized vulnerable groups. However, most focus groups described a
situation of many unmet needs, a perspective that was largely confirmed by interviews with UNICEF
and partner staff. Some interviewees attributed this to lack of awareness and incentives among
UNICEF and partner staff to address vulnerability while they were attempting to balance numerous
priorities.
96
111. In Mozambique, there were distributions of crutches and walking sticks, and programmes covered
medical check-up fees for those living with disabilities. There was little evidence from interviews,
focus groups or reporting that UNICEF actively promoted participation in decision-making by
vulnerable groups. In Sofala, C4D-supported partners targeted children with disabilities-related key
messages. UNICEF was planning to launch activities in resettlement sites with partners specialized in
disability
97
but, aside from assessment, implementation had not yet begun by early July.
112. In Malawi, drama clubs supported by UNICEF consulted with people with special needs and
included relevant messages during their performance. UNICEF supported the Malawi Council for the
Handicapped to consult people with special needs in displaced communities in Chikwawa district.
Examples observed by the team were in the WASH sector, where some piloting on sanitation was
seen in Malawi and latrines were designed for people with disability in schools in Zimbabwe.
Child protection
113. In Mozambique, addressing child protection was particularly challenging in the context of the
cyclone due to the scale of displacement, continuous movements of the population and difficulties in
accessing certain areas. After the cyclones hit, there were concerns about children being separated
from families during search and rescue operations and subsequently during relocations from schools
and public buildings into temporary accommodation sites and then into government-designated
resettlement sites. UNICEF support a coordinated awareness campaign during returns and
resettlements on prevention of family separation and available mechanisms for family tracing and
reunification. This campaign reached 7,012 people in affected communities. Additionally, UNICEF,
96
PCAs reviewed did not contain any obligation for partners to consider disability. UNICEF Zimbabwe had a PCA with a partner
organization specifically focused on providing specialized protection and rehabilitation services for children with disabilities
from 1 May to 21 July 2019, but the team could not find any reports or other evidence of implementation.
97
One of the partners, Light for the World, was leading the disability working group, which falls under the Protection Cluster in
Beira.
49
through the Safe Spaces programme, reached 68 per cent of its HAC target of 20,000 children in
need of psychological support by mid-2019.
98
114. UNICEF had been supporting harmonization of case management systems and fees for case
management workers. This was a work in progress at the time of field visit of the RTE team, with
PCAs being signed. There was little evidence of service provision for children with special needs or
unaccompanied children. The only psychological support that was mentioned was being offered by
Oxfam in Guara Guara camp in Buzi, Sofala Province and Muwawa in Dombe in Manica Province.
115. In Malawi, UNICEF provided child protection support in fifteen camps for the displaced through
the local NGO Yoneco as well as Save the Children; however, focus group discussions reported that
many displaced people living in other camps remained uncovered.
99
Communities had several ways of
reporting cases of abuse. These included Yoneco’s helpline services and Save the Children’s toll-free
line, child protection committees and complaint committees in communities. According to UNICEF
and partner key informants, the main challenge was follow-up of cases of abuse; many cases were left
unresolved. Overall, the child protection response took time to mobilize. This, together with the high
mobility of the population, largely accounted for the relatively low coverage of child protection
interventions. By mid-2019, UNICEF had only reached 13 per cent of its HAC target of 150,000
children receiving psychosocial support through Safe Spaces. UNICEF also reached 14 per cent of
the HAC target of 7,000 people reached by sexual and gender-based violence prevention and
response services.
100
116. In contrast to the other two countries, the child protection response in Zimbabwe was relatively
rapid and achieved good coverage, attributed in part to the limited area affected by the cyclone. A gap
in the initial response, acknowledged by several key informants, was the failure to address the risk of
children being abused by military officers and humanitarian workers during the initial phase of the
response. The RTE did not find direct evidence of such cases, although this issue was raised by a
number of key informants.
101
By mid-2019, UNICEF Zimbabwe had reached a total of 49,238
vulnerable boys, girls and adolescents with child protection services, exceeded its initial HAC target
of 40,000. 682 unaccompanied and separated children were also supported in accessing appropriate
care and child protection services, 45 per cent of the HAC target of 1,500.
102
Focus group discussions
with communities confirmed satisfaction with quality and coverage of psychosocial support to
children and teachers but pointed to widespread unmet psychosocial needs among those parents who
remained displaced who had lost their houses and/or livelihoods.
KQ 4: Resilience and preparedness
KQ 4
To what extent is UNICEF’s response contributing to longer-term goals of enhancing prevention of
future emergencies, mitigation of negative effects of future natural hazards (resilience/sustainable
solutions) and preparedness?
Summary response to KQ 4
98
UNICEF Mozambique, Mozambique Humanitarian Situation Report, Cyclone response mid-year sitrep #12: March – July 2019.
Percentage achieved is based on initial HAC targets.
99
These figures are based on data from the Master Table Data and interviews with Save the Children and YONECO during the
RTE field visit at the end of June. UNICEF Malawi subsequently reported supporting a total of 808 centres in 73 camps,
although the RTE team was not in a position to independently verify these figures.
100
UNICEF Malawi, Malawi Humanitarian Situation Report, July 2019.
101
The low levels of reporting of potential and actual sexual abuse by the military must also be seen in the context of the human
rights situation in Zimbabwe, where civil society and the general public fear repercussions for criticism of the government.
102
UNICEF Zimbabwe, Zimbabwe Humanitarian Situation Report: Multi-hazard, mid-year sitrep – 30 June 2019. Percentage
achieved is based on initial HAC targets.
50
• UNICEF Malawi’s response plan was the only example in the three countries of an overall plan
that specifically recognized the need for communities to restart livelihoods without waiting for
large-scale recovery programmes. UNICEF country offices in Mozambique and Zimbabwe
relied mainly on sector-specific phased approaches to early recovery; at an organizational level
they focused on preparations for long-term recovery, which was expected to take time to gain
momentum.
• Cash transfers had a potentially important role to play in early recovery. Malawi was the only
country where UNICEF adapted its safety net programme to channel funds to vulnerable
individuals during the initial three months (scale-up) phase of the response. UNICEF and WFP
launched a joint cash voucher pilot in Mozambique in August 2019 for vulnerable families; it
was planned that this would feed into a cash transfer scheme as part of a national social
protection programme.
• UNICEF’s longer-term programming in each country has been conducive to promoting
resilience. Government counterparts viewed UNICEF as a key partner not only in terms of
responding to this disaster but also for building their capacities during recovery and continuing to
support them during future disasters.
4.1 Incorporation of recovery considerations into the response
117. As described under KQ 1, early recovery operations were not fully synchronized with the needs of
affected communities, most of whom had already started their recovery within weeks after the
cyclone. Since it was projected that World Bank-supported recovery programmes would not start
until early 2020 in Mozambique and Malawi, this left a significant transition period when UNICEF
and other humanitarian agencies had to fill recovery needs while continuing to address the
humanitarian needs of the most vulnerable. None of the UNICEF country offices had an updated
overall strategy that specifically covered this transition period, although such strategies existed at the
cluster/sector level. UNICEF Malawi’s resilience-based programming provided a suitable framework
to facilitate the transition between relief, recovery and development, although some UNICEF’s
partners there felt they could benefit from more specific guidance.
118. Resilience was integrated in UNICEF’s country programme strategies for each country. During the
RTE field visit, it was already evident that UNICEF would have a key advisory and implementation
role during future large-scale five-year recovery programmes supported by the World Bank and other
donors. At the same time, it was clear that these large-scale programmes will take some time to gain
momentum. UNICEF Malawi’s response plan was the sole example from the three countries that
recognized the pressing need for affected communities to restart their livelihoods as soon as possible
without waiting for large-scale recovery programmes to take effect; the country office accordingly
included $700,000 in the response plan to support early recovery.
119. Intervention strategies for WASH in each country were an example of a phased approach from
emergency to recovery. From the onset of the emergency response, WASH strategies included
rehabilitation of damaged water piping systems in urban areas and damaged boreholes and promoted
community-led total sanitation. Other sections’ efforts at longer-term approaches were not helped by
short-term PCAs that provided incentives for partners to focus mainly on humanitarian-type
interventions without necessarily considering the long term (e.g., engagement with government
counterparts and incorporation of disaster risk reduction components).
Hygiene promotion – an example of building back better in Manica Province,
Mozambique
Focus group discussions conducted in a village where residents had to evacuate for several days
due to flooding found that the number of latrines had doubled in the aftermath of the cyclone.
Two thirds of households had them, double the number who had them prior to Cyclone Idai.
The focus groups said that no material assistance had been provided; instead, the increase
51
appeared to be the result of UNICEF-supported hygiene awareness initiatives.
120. Education efforts primarily focused on humanitarian interventions: providing School-in-a-Box and
school kits and setting up temporary learning spaces. At the same time, the section positioned itself to
participate in longer-term recovery programmes, including construction of hazard-resistant schools.
121. A specific focus of child protection staff in each country was engaging in inter-agency efforts to
prevent and respond to sexual abuse in communities affected by the cyclone. UNICEF supported
these efforts by training facilitators and establishing child protection committees. This was thought to
have generally increased awareness around PSEA and sexual and gender-based violence that was
expected to extend into the recovery phase. In Mozambique and Malawi, it was anticipated that the
inter-agency C4D used during the response described above would help transform behaviours in the
longer term.
122. Cash transfers could have potentially played an important role in early recovery in all countries. In
Mozambique, in August 2019 UNICEF and WFP launched a joint cash voucher pilot for 20,000
vulnerable families. This was part of an advocacy strategy aimed at overcoming government
misgivings about cash transfer as a humanitarian tool. It was planned that this programme would
eventually feed into an unconditional cash transfer scheme as a component of a national World Bank-
supported social protection programme.
103
Malawi was the only country where the RTE team found
that UNICEF had adapted its long-term safety net programme in place prior to the cyclone to
channel transfers to vulnerable individuals directly affected by the cyclone.
Recovery and resilience planning and practice by UNICEF Malawi
Following the flooding in 2015, UNICEF Malawi made a concerted effort to learn from and
address numerous shortcomings experienced during the response.104 In addition to identifying
and mitigating critical risks to achieving programme outcomes, including within the framework
of the National Resilience Strategy, UNICEF Malawi recognized the importance of creating
incentives and accountability at a community level.105 To ensure that concepts to improve
resilience were put into practice, UNICEF Malawi planned and budgeted for an evaluation of
their humanitarian programming in 2020 with the specific purpose of providing the evidence
base for the informed transition from humanitarian action to recovery and development using
relevant UNDAF outcomes as benchmarks.106 Findings from this RTE confirmed there had
been considerable improvement (despite some remaining gaps) in timeliness and quality of
emergency response compared with the 2015 response, including in the area of transitioning to
early recovery.
123. Government counterpart ministries in all countries saw UNICEF as a key support, not only in
responding to the disaster but also helping build their capacities for recovery and for responding to
future disasters. UNICEF made various efforts to support reintegration of displaced communities in
their places of origin, although resource limitations meant that coverage was not complete. A gap in
early recovery common to all countries was the lack of preparedness planning for displaced
populations in hazard-prone areas who had to be permanently resettled to areas that were not
their places of origin. This resulted in a lack of clarity about UNICEF’s policy and value-added in
103
UNICEF, ‘WFP and UNICEF launch joint voucher programme to support communities affected by Cyclone Idai’, press
release, 21 August 2019. UNICEF Mozambique played an important role in leveraging donor funds for post-emergency
cash programming based on the joint strategy developed with the World Bank and WFP.
104
Wasunna, N. and E. Tsoka, UNICEF Malawi country office - Internal Lessons Learned from Malawi Floods, 2015.
105
UNICEF Malawi, Country Programme Document 2019–2023, 2018.
106
UNICEF Malawi, Costed evaluation plan – Malawi – UNICEF country programme of cooperation, 2019–2023, 2018.
52
these contexts.
107
124. No plan was in place in Zimbabwe to address the issue of displaced communities in Chimanimani,
with the exception of a private sector-led relocation plan
108
comprising a model town of 500 houses
approximately 60km away from the district. The approaching rainy season was raised in focus group
discussions and by key informants as a source of significant concern for the safety of communities
still residing in camps.
4.2 Contribution to improving the resilience of local government systems
125. UNICEF’s emergency response was seen by UNICEF interviewees as contributing to improving the
resilience of government by harnessing and heightening the intensity and the pace of partnerships
which existed prior to the emergency. From the perspective of many poorer rural communities, needs
have not changed significantly due to the impact of the cyclone.
109
For example, WASH interventions
included development of more sustainable solar-powered water systems in communities hosting
displaced people where many were expected to remain in the long term. C4D supported awareness-
raising campaigns to prepare for future disasters through to inform communities about selecting
settlement areas, building resistant structures and health risks during disasters. UNICEF planned to
support the governments’ plans to strengthen early warning systems to mitigate the impact of similar
disasters in the future.
126. In Mozambique, UNICEF and UN HABITAT had developed a pilot programme for resilient
schools that had already been approved by the government prior to the emergency. This initiative was
expected to be expanded during the long-term post-cyclone recovery programme. As described in the
Innovations section (above), UNICEF Mozambique’s work with INGC to improve its data
management and visualization has the potential to significantly increase capacities within government
agencies.
127. UNICEF Malawi’s long-term multisector programme provided the clearest example of a
resilience-based programme design that includes strengthening response capacities of government
counterparts so that they can scale up when required. Malawi’s response strategy for Cyclone Idai
could thus be viewed as a component of its longer-term strategy, with some additional external
support provided by the L2 declaration. A challenge in Malawi was that, although there was a
consensus that DoDMA was better prepared than during the 2015 flood response, its capacity was
still limited.
128. In Zimbabwe, apart from UNICEF support to early recovery, UNICEF has tended to focus on
preparations for long-term recovery which took time to gain momentum.
110
KQ 5: Coordination
KQ 5
How effectively and efficiently has UNICEF coordinated its response both internally and externally
(with key actors such as other UN agencies, CSOs and development partners, national and local
governments)?
129. This evaluation question looks at how UNICEF balanced its internal coordination with cluster lead
107
This was not unique to UNICEF. Based on available evidence and consultations, none of the other international agencies had
included resettlement sites in their preparedness planning.
108
UNHCR, Tropical Cyclone Idai Update: UNHCR response in Mozambique, Malawi and Zimbabwe, July 2019.
109
A similar conclusion was reached in the evaluation of the response to the 2007 floods – Cosgrave et al., 2017.
110
UNICEF Zimbabwe hosted a lessons-learned workshop in July for provincial-level government staff on the theme of early
recovery planning, but no evidence of concrete actions was seen.
53
responsibilities,
111
how the organization used its comparative advantage, how it set up operational
hubs in disaster-affected areas, and how it supported the government’s coordination role in the three
countries.
Summary response to KQ 5
• Internal coordination between the UNICEF country office and the operational hub was good in
Zimbabwe. In the two other countries, the effectiveness of coordination varied by sector and
was hindered by rapid turnover of surge staff. Internal and external information management was
a constant challenge.
• The humanitarian community recognized UNICEF as a team player that provided significant
support to the coordinated government-led response and inter-agency initiatives.
• The effectiveness of coordination with implementing partners and cluster members correlated
closely with the effectiveness of each sector’s internal coordination. Coordination of nutrition
clusters faced specific challenges because government coordination systems treated nutrition as a
subsector of health.
• Most clusters lacked adequate information management capacity, which had detrimental effects
on the quality of activities and results achieved.
• The relatively good performance of UNICEF-led clusters and working groups contributed to
imbalances in levels of assistance compared with some sectors not led by UNICEF, such as
shelter; this had a significant impact on children’s well-being.
5.1 Balancing UNICEF internal operations with cluster leadership
Internal coordination
130. Internal coordination between the UNICEF country office and the operational hub was good in
Zimbabwe for all sectors. In the two other countries, the effectiveness of coordination varied by
sector. UNICEF internal coordination in WASH worked well at national, provincial and district
levels; the same coordination tools were used, and there was an effective information management
system reinforced by frequent visits to the field by senior staff. Education and child protection
experienced more challenges with coordination because these areas of intervention were not
prioritized, and because of other factors described under KQ 2 (above). C4D was relatively well-
coordinated, largely because of its cross-cutting nature and constant interaction with different sectors
and clusters. Apart from C4D, however, there were a number of gaps in UNICEF’s intersectoral
coordination in Mozambique and Malawi due to the short duration of surge deployments; what’s
more, confusion about reporting lines gave rise to the perception that UNICEF sections were
working in silos.
131. Information management, both internally within UNICEF and externally with partners, was a
constant challenge in Mozambique and Malawi due to a combination of fluid population
movements, gaps in assessment data and other factors described in detail under KQ 1 and KQ 2
(above). In Zimbabwe, UNICEF information management was viewed positively, notwithstanding
information gaps during the initial week of cyclone response, when it was not possible to access
impacted areas. In Mozambique, UNICEF adapted the 4W matrix coordination tool for monitoring
purposes to retrieve operational presence, outputs, targets and achievements. The potential for HAC
111
In line with the scope of the TOR for this RTE, it was agreed during the inception phase that since Clusters are a focus of the
Operational Peer Review and Inter-agency Humanitarian Evaluation for the response in Mozambique, this UNICEF RTE
would mainly focus on internal coordination and limit its analysis of UNICEF’s coordination/cluster responsibilities to how
well it balanced these with its internal operations.
54
targets to change during Humanitarian Response Plan and HAC revision processes could affect the
measurement of achievements versus targets; so, to maintain reasonable accuracy monitoring would
have to take into account movements of affected population and changing needs.
External coordination
132. External key informants recognized UNICEF commitments and actions as a team player that
provided fundamental support to a coordinated government-led response. This commitment
translated into not only investments to fulfil UNICEF’s lead coordination roles and responsibilities,
but also strong support for inter-agency initiatives in areas including PSEA, joint multisectoral
assessments and, in Mozambique, agreement that the UNICEF Representative should continue to
act in the Humanitarian Coordinator/Resident Coordinator role he had occupied since June 2018
instead of only focusing on UNICEF operations.
133. The effectiveness of coordination with implementing partners and cluster members
correlated closely with the effectiveness of each sector’s internal coordination. WASH
implementing partners and cluster members in all three countries expressed high levels of satisfaction
with UNICEF’s performance. UNICEF’s health and nutrition sections worked closely with
government structures and, although this caused some delays, UNICEF supported interventions that
mirrored government coordination protocols and eventually achieved wide geographical coverage.
Implementing partners and cluster members involved in other sectors provided mixed feedback.
Coordination of nutrition clusters faced a specific challenge because government coordination
systems treated nutrition as a subsector of health; as a result, nutrition efforts had to compete with
health-related initiatives that were seen as more of a priority.
134. UNICEF did not strictly adhere to cluster coordination guidance. Cluster coordinators reported to
section heads rather than directly to the Deputy Representative, which made it more difficult to put a
‘firewall’ between UNICEF priorities and the interests of cluster members. This was made even more
challenging in places where UNICEF staff were ‘double-hatted’, combining cluster coordination
responsibilities with UNICEF programming leadership. Most clusters lacked adequate information
management capacity, which had detrimental effects on the quality of outputs, including 4W
reporting.
135. In Mozambique, UNICEF led the development of a gaps analysis matrix for the WASH sector,
along with a mapping of resettlement areas and listing of boreholes marked for rehabilitation in
Manica Province and Sofala Province.
112
WASH cluster standards were developed during 2016 with
the government and were already known to partners, including standardized hygiene kits that had
been field-tested with communities. The WASH section developed technical SOPs, technical
guidelines and a strategy with agreed indicators that are all uploaded onto the online WASH platform.
Apart from a gap in handover between cluster coordinators in Cabo Delgado, overall cluster
coordinators received good feedback from members.
136. The education cluster at a national level in Mozambique was seen by its members as providing good
service. At a field level, the cluster was hampered by transitions and gaps in leadership which, among
other things, resulted in a confusing array of tools and templates despite a co-lead agreement with
Save the Children that should have prevented such problems.
137. In Malawi, clusters were generally perceived as useful by cluster members and government
counterparts, aside from the rapid personnel turnover within some clusters. There were also some
challenges resulting from frequent revision of UNICEF’s reporting templates.
138. In Zimbabwe, UNICEF played a leading role in the initial rapid multisectoral assessment during the
first week. UNICEF-led clusters were viewed as somewhat slow to establish (around three weeks
after the disaster); this contributed to confusion during the initial response, especially since the local
112
Assessments by UNICEF and other WASH cluster members achieved high coverage and helped to fill gaps in assessment data
for other clusters in Mozambique.
55
government had little experience in coordinating international agencies in large-scale emergencies.
Nonetheless, several different cluster members and government counterparts commented on the
marked improvement in overall coordination following UNICEF’s arrival.
5.2 Considerations of comparative advantage for response and recovery
139. A combination of UNICEF’s leading role in multiple coordination mechanisms, its wide geographical
coverage and strong operational capacity and the fact that requests for UNICEF and UNICEF-led
clusters amounted to around a third of the total requirements for the response translated into
UNICEF’s strong influence on the overall response. Since UNICEF viewed itself as a part of a
collective effort, there were a number of examples where the organization used its comparative
advantage to benefit the wider humanitarian community (e.g., sharing WASH cluster data to fill in
assessment data gaps in Mozambique and playing a key supporting role during the initial
multisectoral assessment in Zimbabwe).
140. At the same time, UNICEF’s influence resulted in some imbalances in levels of assistance. In
Zimbabwe, for example, parents and teachers expressed appreciation for the psychosocial support
provided for their children and students; at the same time, however, they highlighted the lack of
similar types of support for adults who had lost homes, family members and livelihoods. UNICEF’s
child protection section in the first six weeks supported psychosocial first aid aimed at reaching both
children and adults; later, however, UNICEF narrowed its focus to children and young mothers due
to limited capacity. Similarly, response gaps in the areas of food security, livelihoods and shelter –
sectors where UNICEF did not have a lead role – adversely impacted the lives of children.
141. The RTE team received consistently positive feedback from all government authorities interviewed in
all three countries about UNICEF’s support for these governments’ lead coordination roles. With the
exception of Malawi, which in 2015 experienced severe flooding in the same areas affected by
Cyclone Idai in 2019, local officials had very little prior experience coordinating an international
response to a major disaster. There was widespread appreciation for UNICEF’s technical expertise,
its proactive engagement with the government and its flexibility in providing logistics support for the
governments’ lead roles during the response.
142. In Mozambique, government counterparts felt that UNICEF provided overall good support. In
Cabo Delgado, the main challenge for UNICEF was finding Portuguese speakers who could engage
at a strategic level. The Department of Health worked directly with UNICEF during the week-long
vaccination campaign. The effectiveness of child protection, education and WASH clusters was
reduced by incomplete handover processes.
143. In Malawi, there was also overall good appreciation on the part of the government for UNICEF
support in all sectors. UNICEF staff, however, felt that coordination with DoDMA could have been
better. UNICEF staff partially attributed this to DoDMA’s centralized structure.
144. In Zimbabwe, government authorities consistently expressed their appreciation for the way
UNICEF coordinated with the Civil Protection Unit and with the districts in all intervention sectors.
Implementing partners’ perspectives on UNICEF’s coordination were also very positive, with
partners working together as a team.
5.4 Coordinating setup and operation of hubs
145. UNICEF initially used temporary office solutions, mainly in hotels, when setting up hubs. The
organization only established a long-term UNICEF presence in Beira (Mozambique) in July 2019,
after deciding that it was needed for longer-term programming. UNICEF deployed administrative
staff and streamlined internal procurement and administrative systems to aid setup. UNICEF staff
deployed during that period felt that infrastructure was sufficient for this approach, although living
and working conditions were not optimal during the first few weeks.
113
Transaction costs for
113
UNICEF staff in Mozambique and Malawi cited this as a reason for the short 2-week surge cycles.
56
UNICEF were relatively high in Beira due to short surge rotations and frequent missions.
KQ 6: Integration of gender
KQ 6
To what extent has gender been integrated in needs assessments, planning, implementation, monitoring
and reporting of the response, as well as in recovery planning?
146. This evaluation question examines the gender and equity dimensions of the emergency, including the
extent to which gender was integrated into UNICEF’s response.
Summary response to KQ 6
• The gender and equity dimensions of the emergency response were comparable in all three
countries, with a range of gender and equity gaps identified via gender analyses carried out by
other humanitarian agencies.
• Some gender mainstreaming was evident in WASH, education, C4D, health and, to a limited
extent, child protection. UNICEF country offices did not conduct a gender analysis of their
overall programmes or made systematic efforts to use existing gender analyses in programme
design and implementation.
• The team found limited evidence women’s participation influencing UNICEF decision-making
processes.
• UNICEF played leading roles in establishing inter-agency call centres in Mozambique and
Malawi to mitigate PSEA.
6.1 Gender and equity dimensions of the disaster
147. The gender and equity dimensions of the emergency response were similar in all three countries,
including several gender and equity gaps. A contributing factor was that all three countries scored
poorly in the UNDP Gender Inequality Index,
114
with relatively high rates of illiteracy, child marriage,
teen pregnancies, HIV infection rates and unsafe abortion practices. Specific gender and equity
issues highlighted in rapid assessments by other humanitarian organizations
115
and post-disaster needs
assessments included negative coping mechanisms, such as exchanging sex for relief items or turning
to prostitution as a coping strategy. Issues raised in these assessments and during focus group
discussions in affected communities included increased risk of sexual and gender-based violence,
violence against children in schools, child marriage and child labour due to loss of livelihoods and
influxes of large numbers of people from outside the community. There was also higher likelihood of
girls dropping out of school due to caregivers’ loss of livelihoods and the adults’ preoccupation with
the challenges of rebuilding their lives. Assessments found disproportionate numbers of women
among displaced populations, with a number of women reporting being abandoned by husbands.
116
6.2 Integration of gender into UNICEF interventions
148. UNICEF country offices did not conduct a gender analysis of their programmes or made
systematic efforts to use existing gender analyses in programme design and implementation.
Focus group discussions with affected communities found that, apart from being represented in the
114
UNDP, Human Development Report 2015: Work for Human Development, 2015. Zimbabwe, Mozambique and Malawi are ranked
128, 138, 148 respectively out of a total of 160 countries.
115
See, for example, CARE International Rapid Gender Assessments for each country.
116
In Malawi, 63 per cent of the population in shelters for the displaced in Machinga, Mangochi, Balaka and Zomba were
women. Source: Government of Malawi, Post-Disaster Needs Assessment Malawi, 2019.
57
camp committees, there was little evidence of women influencing UNICEF decision-making
processes.
149. In Mozambique, UNICEF’s response was constrained by the lack of disaggregated data,
despite evident needs. UNICEF rolled out a policy addressing sexual and gender-based violence, led
the inter-agency PSEA network and supported the inter-agency Linha Verde call centre. In WASH,
gender was integrated as a cross-cutting issue through segregation of sanitation facilities by gender
and distribution of dignity kits. During Health Week, gender was mainstreamed by promoting men’s
involvement in reproductive health services, HIV testing and counselling and support for nutrition
work. However, these activities were not always implemented in a gender-responsive way using
relevant assessment data.
117
UNICEF’s gender specialist trained volunteer teachers, officers and
education and WASH partners to raise gender awareness. Dignity kits were distributed, though they
were perceived in women’s focus group discussions as insufficient to cover needs. Child sexual
exploitation was reported to be prevalent and was associated with harmful traditional practices.
150. In Malawi, an inter-agency rapid assessment tool developed with UNICEF’s support included
gender and age disaggregation, but it proved challenging to obtain disaggregated data from camps for
the displaced and use this data to appropriately influence interventions. The RTE team found
evidence of gender mainstreaming
118
in WASH, education, C4D and, to some extent, child
protection, with some mainstreaming gaps identified for adolescent girls. An inter-agency complaints
and feedback mechanism with two helplines set up by UNICEF’s partner Youth Net and Counselling
(YONECO) provided a way for communities to report cases of PSEA and sexual and gender-based
violence. Cases were referred to relevant services, though gaps were reported in follow-up. Training
in gender awareness (including addressing sexual and gender-based violence, child protection and
PSEA) was provided to volunteer teachers deployed to schools for displaced students. UNICEF also
worked with the Malawi Police Service to enhance community policing in flood-affected areas as
part of the early recovery phase. C4D supported awareness-raising on sexual and gender-based
violence and PSEA through drama and communications with affected communities.
151. In Zimbabwe, gender was integrated to some extent into UNICEF interventions. Mosquito nets
were distributed along with sanitary pads for girls and women. Expectant mothers were airlifted to
more accessible areas near highways, where they received support from UNFPA. UNICEF provided
no specific interventions for girls, though UNDP helped with payment of school fees for vulnerable
girls and training on HIV and AIDs in schools. In Chipinge, provision of sanitary pads/dignity kits
for girls and women and free transport to the clinic for pregnant women were among the services
targeting women. Focus group discussions in Chimanimani district reported that families with
adolescent girls had rented houses in the neighbouring communities because it was considered unsafe
for young girls to stay in the camp. The role of the army during the initial phase of the emergency
(until road access was restored) raised concerns about PSEA.
Lessons learned
This section presents selected key lessons learned that will be useful for UNICEF staff when revising
preparedness approaches or implementing a response during similar emergencies, whether in these three
countries or elsewhere. A more complete list is in Table 7, which lists the main factors that contributed
to, or inhibited, a timely and effective response.
117
In one camp visited by the team, an equal number of bathing shelters had been built for men and women, even though there
were many more women than men in the camp.
118
‘Mainstreaming’ in this context uses the CCCs as a reference, i.e. “Promoting gender equality in humanitarian action is most
effectively achieved by ensuring that the assistance and protection provided are planned and implemented to benefit girls,
boys, women and men, in line with an analysis of their rights, needs and capacities”.
58
152. This quick-onset large-scale disaster underlined the importance of preparedness measures. Key
examples of preparedness that UNICEF demonstrated in this emergency response (to greater or
lesser extent in the three countries and in different sections) included prepositioned stocks in strategic
locations and standby PCAs with experienced partners. Also important were joint preparedness with
government counterparts, strong donors and strategic partnerships with other humanitarian agencies
that ensured UNICEF ability to quickly access and expand coverage in affected areas. Rapid
deployments, both from within countries and from the region and elsewhere, of trained and
experienced surge personnel to support UNICEF and cluster operations also provided key support.
Specific examples of all these positive practices that could potentially be used to develop case studies
of good practice to encourage replication are as follows:
a) Supporting governments’ leadership and coordination of the response with innovative
solutions to assessment and communicating needs. In Mozambique, this took the form of
UNICEF offering their data visualisation IT platform as a coordination tool to INGC. To carry
out initial assessments in Malawi, UNICEF complemented the government’s remote-sensing
capacities and the Malawi Red Cross with its drones and technical staff. In Zimbabwe,
UNICEF played a leading role in the initial multisector rapid assessment that helped overcome
the lack of a pre-agreed assessment format.
b) UNICEF Malawi’s country programme strategy has made significant progress in integrating
resilience in a way that promotes practical application based on lessons learned
119
during
previous disasters. This included a cash-based social protection system that was expanded
after the cyclone to cover increased numbers of vulnerable people. The country programme also
helped strengthen the capacities of the government and the wider humanitarian community in
Malawi by partnering with research institutions to improve use of early warning and
anticipatory action triggers. Because of this pre-existing approach, UNICEF Malawi was able
to transition relatively rapidly to early recovery in the aftermath of the cyclone.
c) The WASH section’s ‘shadow’ emergency team in Mozambique, was such that staff
working on longer-term activities were able to quickly transform into a deployable response team
thanks to their previous emergency training and experience of working in other responses. This
shadow team was supported by good preparedness planning, strategically positioned contingency
stocks and pre-agreed systems worked out jointly with government counterparts and partners.
While this kind of preparedness is critical to mobilizing a rapid and effective response to quick-
onset emergencies, in-country capacity was particularly important in this context due to the
challenge of quickly deploying experienced surge staff with Portuguese language skills.
d) UNICEF’s national emergency coordinators played pivotal roles during the response in both
Mozambique and Zimbabwe, thanks to their knowledge and experience and to their pre-
existing networks, which recognized and valued their technical leadership.
e) Outcome monitoring systems were adapted to the humanitarian context, mainly in the
form of post-intervention monitoring; these were used by some sections in UNICEF Zimbabwe
and by some of UNICEF’s partners and provided insights into the coverage and effectiveness of
assistance.
f) Deployment of in-country and international surge capacities by UNICEF Zimbabwe
optimized value-added.
g) Senior UNICEF leadership at a regional and global level committed to implement L2/L3
scale-up protocols. The combination of early visits by senior decision makers, rapid allocation
of significant funding from reserves on a no-regrets basis and consistent support to UNICEF
country offices help ensure they had the necessary surge capacity, resource mobilization support
and strategic guidance.
119
Wasunna, N. and E. Tsoka, Internal Lessons Learned from Malawi Floods, 2015.
59
153. One dimension of cyclone response that holds a lesson for UNICEF’s future work is ensuring that
resettlement of displaced populations in areas that are not their places of origin is
incorporated into preparedness planning. Application of this lesson should result in a more coherent
position for UNICEF in terms of supporting these affected populations and should help guide
operations and advocacy based on pre-agreed minimum standards.
60
Conclusions and recommendations
154. This chapter presents conclusions and recommendations emerging from findings and analysis during
this RTE. They are structured as follows:
• An overall statement for the overall response to the crisis; and
• Conclusions linked to corresponding recommendations based on an analysis of evidence
collected in the evaluation.
155. As described in the Methodology section, emerging conclusions were discussed with stakeholders
during an inter-agency validation workshop in Mozambique, and in debriefings at the end of each
country visit and subsequently with ESARO and headquarters staff. The conclusions below have
considered feedback from these interactive sessions.
OVERALL CONCLUSIONS
UNICEF was a major international actor during this cyclone response and made significant
contributions to government-led responses to the cyclones and early recovery efforts in all three
countries. These contributions included collaboration with the government authorities and partner
agencies to ensure that cholera, which is endemic in all three countries, did not create a ‘double-
disaster’ event. UNICEF’s achievements can be attributed to a combination of pre-existing
relationships with government in multiple sectors; effective inter-agency coordination; UNICEF’s
willingness to advance significant reserve funding to kick-start the response; programming that
integrated preparedness and resilience; and in-country, regional and global surge mechanisms that
helped to fill capacity gaps. UNICEF provided key support n fulfilling their lead coordination roles
to local governments unaccustomed to dealing with such a large-scale disaster. UNICEF fulfilled
its IASC cluster responsibilities relatively well and provided important support to inter-agency
initiatives, including PSEA systems and initial multisectoral assessments.
UNICEF should view this as a timely and good-quality response that was well organized, with
strategic and timely support provided by both the regional office and headquarters divisions.
Despite this, there were some important gaps in the organization’s response. UNICEF was well-
positioned after the emergency phase had ended to play a critical role in long-term recovery, but
the transition period from emergency to early recovery lacked coherence in Mozambique and
Zimbabwe because sectors/clusters were mostly operating independently, in contrast to the initial
emergency phase when there was robust overall coordination. The disaster also left women and
adolescent girls more vulnerable and exposed to sexual exploitation and increased the vulnerability
of people living with disabilities. While these risks were recognized in UNICEF strategies, what
was lacking was a systematic effort to put these commitments into practice based on relevant
analysis and incentives.
156. Recommendations listed below are targeted specifically at UNICEF headquarters, ESARO and
UNICEF country offices, although many are also relevant for UNICEF’s partners. Most of these
recommendations are reflected in the ‘real-time’ action plans developed in consultation with
UNICEF country offices at the end of the field visit (Annex 2). Recommendations are grouped into
three categories, 1) emergency operations, 2) cross-cutting issues, and 3) monitoring, evaluation and
information management. Timeframes have been defined where relevant.
61
Emergency operations
CONCLUSION 1.
Preparedness and anticipatory action
Preparedness planning proved to be a significant asset that strengthened UNICEF’s response in all
three countries. Specific areas to tackle to improve preparedness include lack of ‘shadow’ response
teams, lack of cash-based assistance as an intervention option, more systematic consideration of
early recovery to respond to community needs in real-time and planning appropriate support for
resettlement of communities in areas that are not their places of origin.
Based on findings from KQ1, KQ2, K3 and KQ4
R1. Improve early warning and preparedness planning based on lessons learned and include
thresholds that trigger anticipatory action
120
prior to a disaster.
UNICEF country offices, with support from ESARO and UNICEF headquarters
• Support strengthening governments’ early warning systems based on lessons learned.
• Develop thresholds for anticipatory action
121
and associated response protocols.
• As part of staff development strategies, UNICEF’s country offices should create ‘shadow’
emergency teams composed of staff involved in longer-term work who can rapidly switch gears
and transform into an effective emergency team. UNICEF’s WASH section provided a good
example of this, with staff members who were empowered by prior emergency training and
relevant experience and joint preparedness with partners.
• Further improve UNICEF’s preparedness, notably by incorporating accountability to affected
populations and cash-based assistance into joint preparedness planning with partners
122
and
updating UNICEF response plans in a timely way to adapt to community needs for transition
into recovery.
• In UNICEF and inter-agency preparedness plans, include scenarios for resettlement of displaced
populations in sites that are not their places of origin. Agree with governments on minimum
standards that can be used to guide operations and advocacy.
• Use relevant lessons from this response to strengthen preparedness planning.
CONCLUSION 2.
Transition from emergency to early recovery
Due in large part to its long-term programming in multiple sectors and strong relationships with
governments, UNICEF is positioned to play a major role during the recovery in all three countries.
The major impacts of the cyclones were in areas with relatively high levels of poverty, and
UNICEF’s role is not only to support disaster recovery, but also to ‘build back better’ and raise
standards in communities. Resilience could be more systematically incorporated into intervention
120
See Annex 7 for a definition and timeline for anticipatory action.
121
The term ‘anticipatory action’ here is used as a concept developed by FAO, ODI, and the Inter-Agency SOPs for Early Action
to El Niño/La Niña Episodes as “…an activity taking place between an early warning trigger or a high-probability forecast and the actual
occurrence of the corresponding disaster in order to mitigate or prevent the humanitarian impact of the anticipated disa ster”. See CERF, ‘CERF
for the Future: Anticipatory Humanitarian Action – Update for the CERF Advisory Group – October 2018’.
122
Through engagement with the Community Engagement and Cash Working groups.
62
design. School near areas at risk could be better designed to accommodate future displacements.
Scenario-planning could inform standards for WASH facilities, shelter and protection. UNICEF,
and the international humanitarian community as a whole, struggled to meet the needs of displaced
populations in resettlement sites that were not places of origin. This can be attributed in large part
to a preparedness planning that did not consider this scenario.123 While this was a challenge for
UNICEF in all three countries, in Mozambique the ad hoc nature of resettlement of over 50,000
people absorbed a considerable amount of resources which, along with the need to respond to
cyclone Kenneth, made early recovery that much more difficult.
Based on findings from KQ2, KQ4 and KQ6
Recommendations targeted at UNICEF country offices
R2. Organize workshops
124
, involving field-based staff from UNICEF and partners, to update
assessments and guide the transition between the humanitarian response and recovery to
ensure that UNICEF optimizes its value-added for affected communities.
• Replicate relevant elements in UNICEF Malawi’s country programme and the phased approach
to facilitate transition promoted by the WASH and health sections in Mozambique.
• Define a clear action plan for building back better (both infrastructure and services).
• Strengthen gender responsiveness in preparedness, response and recovery (see R9 for additional
recommendations related to this).
• Strengthen efforts to address and reach the needs of people with disability. In so doing, the
country offices should strengthen and deepen inter-sectoral work among all programme sections
and ensure attention to children with disabilities and other risks and vulnerabilities.
• Develop advocacy strategies to fill gaps in sectors that impact the welfare of children but where
UNICEF does not have a lead role (e.g. shelter, livelihoods).
• Determine optimal office and staffing configurations for hubs and how to strengthen links
between the country offices and hubs.
• Update preparedness, notably for drought and floods.
• Make use of monitoring data to adapt recovery interventions as needs on the ground change.
CONCLUSION 3.
Value-added of surge deployments
The value of surge deployments in Mozambique and Malawi was decreased by their short
deployment durations. Short-duration deployments caused gaps between deployments, had
negative impact on teamwork, increased transaction costs on UNICEF management and created
inconsistencies in relationships with partners and communities. In Mozambique, there were
123
In Mozambique in particular, it has been common practice to resettle communities living in at-risk areas to other areas
following flood disasters.
124
This was one of the main ‘real-time’ recommendations agreed with UNICEF country offices following end-of-mission
debriefing sessions and described in the Real-Time Action Plans for each country. This recommendation is similar to a
recommendation in Real-Time Evaluation of UNICEF’s Humanitarian Response to Typhoon Haiyan in the Philippines that UNICEF
should “Plan the main ‘recovery’ elements of the programme as a coherent package, based on a re-assessment of needs and
discussion with partners, and produce a recovery plan by the three-month mark…”, Darcy, James et al., 2014, p. 46.
63
insufficient surge human resources staff to cope with the increased workload and a lack of
Portuguese-speaking surge candidates to fill strategic liaison and coordination roles. Zimbabwe
provided a good example of surge that optimized value-added (in terms of deployment length) and
filled critical capacity gaps.
Based on findings from KQ2
R3. Ensure that surge deployments, notably by UNICEF staff, add optimum value to
UNICEF’s response and recovery interventions.
UNICEF country offices
• Develop ‘shadow’ emergency teams
125
within each section in UNICEF country offices by
integrating relevant preparation into staff development plans and investing appropriately. Agree
on standards (training, roster membership), SOPs and annual review (simulation together with
partners followed by updating preparedness plans).
UNICEF headquarters
• Revise regional and global standards and SOPs to optimize the value-added of surge staff,
notably in setting minimum deployment lengths and developing handover processes that add
value.
CONCLUSION 4.
Supply chain tracking
Procurement of relief items was a major component of cyclone response. UNICEF’s Supply
Division provided effective and efficient support to the operation through expedited procurement
and transportation of prioritized relief items. This work was supported by six-week deployments of
surge staff where required. The main area for improvement this RTE identified was improving
tracking systems so that they provide a more user-friendly tool for programme staff to track
procurement in real time; at the same time, this would improve UNICEF’s accountability to
affected populations by giving a more complete picture of the supply chain from request to delivery
to the end user in communities.
Based on findings from KQ2
R4. Improve the efficiency of the supply chain through more effective use of regional
resources and support UNICEF’s accountability to affected populations by using
innovative approaches to extend tracking to communities affected by disasters.
UNICEF ESARO
• Accelerate feasibility study and analysis of South Africa as a major procurement hub for the
region in consultation with UNICEF country offices and in collaboration with Supply Division.
UNICEF headquarters (Supply Division)
• Improve the user-friendliness of reporting and tracking systems to transform them into tools for
UNICEF programme managers and sector heads.
• Use innovative systems
126
so that relief goods can be tracked to the end user in beneficiary
125
The WASH section in UNICEF Mozambique provided a good practice model for this that could be replicated.
126
For example, UNICEF already has considerable experience in use of SMS systems with U-Report.
64
communities to help UNICEF fulfil its commitment to accountability to affected populations. If
this is combined with increased transparency and information-sharing (specifically regarding aid
entitlements and intervention objectives), beneficiary communities will be in a good position to
not only confirm receipt of relief items but also provide feedback on their use.
CONCLUSION 5.
PCA processes in Mozambique (preparedness)
PCA processes in Mozambique for some sectors had high transaction costs and, in some cases,
delayed the response. Contributing factors included lack of standby PCAs for some sectors, notably
education and child protection, lack of longer-term PCAs to support early recovery, and rapid
turnover of UNICEF focal points resulting in a situation where negotiations were constantly
restarting.
Based on findings from KQ1 and KQ2
Recommendation targeted at UNICEF Mozambique
R5. Improve protocols for standby PCA incorporation into preparedness planning to
streamline processes consistently across all sectors during future emergency responses.
Cross-cutting issues
CONCLUSION 6.
Accountability to affected populations
UNICEF can point to some areas of progress towards meeting its commitment to accountability to
affected populations, notably its support of inter-agency PSEA initiatives and protection-related
activities in child protection and health (e.g., sexual and gender-based violence) in all three
countries. However, findings indicate that affected communities have largely been passive
recipients of assistance. Community members heard in focus group discussions were unaware of
what assistance was being delivered and indicated that their participation in assistance operations
had been marginal. UNICEF support to inter-agency complaints and feedback mechanisms in
Mozambique and Malawi was consistent with good practice, although none of the three
UNICEF country offices had internal protocols for following up on complaints or feedback from
communities affected by disasters, whether received from inter-agency CFM or other sources.127
While there was considerable interest among staff in improving accountability to affected
populations, there was relatively little knowledge about how to achieve this in practice. On the
other hand, many of UNICEF’s international NGO partners have made considerable progress
integrating accountability to affected populations into their humanitarian programming, which
offers potential learning opportunities for UNICEF.
Based on findings from KQ1, KQ2, KQ3, KQ4, KQ5 and KQ6
R6. Improve accountability to affected populations to optimize the value-added of UNICEF
operations for disaster-affected communities and fulfil UNICEF’s accountability
commitments.
127
While collective complaints and feedback mechanism are considered good practice, they work most effectively when they can
interface with agencies’ internal systems so that accountability and protocols are clear. See, for example, Inter-Agency
Standing Committee, Guideline: Inter-agency community-based complaint mechanisms –Protection against sexual exploitation and abuse,
2016.
65
UNICEF country offices, including staff with cluster coordination roles
Short-term priorities – within the next six months:
• Organize an AAP orientation session for senior staff and training for programme staff. The aim
is to build capacity and integrate accountability to affected populations into overall country office
strategic plans (including in emergency preparedness plans), with a view to developing an AAP
implementation framework.
• Facilitate discussions and workshops with government, other UN and NGO partners to agree on
collective approaches and other strategic ways of working to improve accountability to affected
populations and fulfil relevant commitments.
128
• Nominate an overall country office focal point for accountability to affected populations (with a
reporting line to the Representative,) as well as focal points and alternates in each unit (including
operations) to ensure that AAP is sufficiently prioritized. Where possible, country offices are
encouraged to recruit staff with AAP-specific tasks (a generic TOR/JD can be shared from the
AAP unit in EMOPS).
• Establish internal UNICEF complaints and feedback systems that can interface with inter-agency
systems (such as Linha Verde in Mozambique) with relevant protocols and guidance to facilitate
follow-up and closing feedback loops with affected communities.
• With partners, develop a joint communication strategy for communities that pro-actively informs
affected populations, including vulnerable groups, about their entitlements and rights, complaints
and feedback mechanisms, and UNICEF’s commitments to put communities at the centre of our
work.
Medium-term priorities – within the next 12 months:
• Incorporate relevant AAP elements into various UNICEF systems and procedures (e.g.,
workplans, partnership agreements, contingency arrangements, monitoring, capacity building and
other strategic plans).
UNICEF headquarters (accountability and community engagement)
• Proactively engage with ESARO and UNICEF country offices in the region to provide guidance,
tools and coaching on how best to position AAP in emergency preparedness and response
actions, while facilitating involvement of partners as a valuable learning resource. This should
also be done for routine programming.
• Routinely conduct AAP trainings for UNICEF staff, cluster coordinators and partner staff to
develop or strengthen preparedness.
129
Such trainings should also be done following a corporate
disaster event. These trainings, based on global AAP training standards and modules, would
benefit from interactive sessions using context-specific examples on participatory planning during
an emergency response, information sharing and collecting and processing complaints and
feedback from communities. UNICEF can directly lead these trainings or do so in in
collaboration with other agencies initiatives.
UNICEF headquarters (Supply Division)
• Review guidance and Standard Operating Procedures from an AAP perspective. Update tracking
128
For example, UNICEF’s AAP Framework, IASC’s AAP commitments, the Core Humanitarian Standard on Quality and
Accountability, accountability frameworks of individual agencies. These workshops should be an opportunity to validate the
revised guidelines and SOPs and to improve them (as needed based on feedback) and promote their use.
129
Training during the response phase may be challenging but should take place by the time early recovery has gained
momentum.
66
systems to measure fulfilment of AAP commitments through to end users, including their
feedback about timeliness and quality of relief items.
CONCLUSION 7.
Gender and equity
UNICEF did not carry out a gender analysis during the response in any of the three countries and
this, along with gaps in sex- and age-disaggregated data, resulted in limited attention to gender-and
equity programming components, limited engagement of women and girls in decision making and
no monitoring of gender gaps in the response.
Based on findings from KQ6
R7. Strengthen gender integration in all phases of future emergencies; preparedness,
response and recovery in planning, implementation and monitoring and evaluation.
UNICEF country offices, with support from ESARO
• Conduct gender and equity assessments in line with the CCCs (or tap into assessments being
conducted by other partners) of the rights, needs and capacities of girls, boys, women and men in
affected populations. This should lead to a more accurate understanding of the situation of girls,
boys, women and men and guide the design of appropriate response interventions.
• Ensure the collection, analysis and use of sex-disaggregated data to understand and address
gender specific needs.
• Strengthen efforts to mainstream gender in all aspects of UNICEF’s response. Ensure gender is
integrated into the work of sectors, in preparedness and response plans, in implementation, in
monitoring and implementation and in reporting.
• Appoint gender focal points in each section whose role is to raise awareness and understanding
of how to mainstream gender-related issues.
CONCLUSION 8.
Value for money of UNICEF interventions
Various studies show that VFM relies on systems and processes that encourage its application
when managing projects, programmes and resources.130 An important step is promoting a shift in
mind sets and organizational culture so that staff are aware of their responsibilities as ‘investment
managers’ with the authority to make decisions on behalf of communities affected by humanitarian
crises to optimize VFM of assistance.
With the exception of the procurement unit, where it is clear that VFM is embedded in their
systems and ways of working, this RTE team observed relatively low awareness among UNICEF
staff of Value for Money in the sections, an indicator that they were not optimizing cost
effectiveness.
130
See, for example, Petrovic, Darko, ‘Mainstreaming “Value For Money” in Development and Humanitarian Programming -
Some practical experience’, guest presentation at the Vienna Evaluation Network, 25 September 2018; Baker, Jock et al.,
Study: How to define and measure value for money in the humanitarian sector – Final report, SIDA, 2013; DFID, DFID Value for Money
in Humanitarian Programming, 2011.
67
Based on findings from KQ1 and KQ2
R8. Promote greater value for money (VFM) awareness among staff and partners while
improving systems to facilitate VFM analysis by UNICEF decision makers.
131
UNICEF country offices, with support from ESARO
• Develop checklists to support decision making by facilitating VFM analysis of unit costs and
intervention options to improve UNICEF’s cost effectiveness.
• Improve feedback loops for outcome monitoring (link with Recommendation 6).
• Encourage implementing partners and cluster member to benchmark unit costs and share
relevant learning to optimize value-added.
UNICEF headquarters
• Improve global systems and relevant guidance to facilitate decision making that systematically
considers VFM.
132
• Facilitate donors’ communication flow on VFM expectations at the global level and agree on
minimum VFM requirements to ease transition between emergency and recovery phase.
Monitoring, evaluation and information management
CONCLUSION 9.
Information management and monitoring
UNICEF’s operations and cluster coordination roles were constrained by a general lack of relevant
data during the first weeks of the response. In Mozambique and Malawi, this contributed to
uneven coverage of affected populations, with the exception of WASH interventions. Lack of
relevant data was a challenge common to all agencies, including national disaster management
agencies. In all three countries, the country offices adapted RapidPro open source online platforms
to support decision making by improving data visualization, not only for UNICEF but for other
stakeholders,133 but this took time to set up. Ultimately, there was limited evidence of its use
outside of UNICEF, apart from in Mozambique, where INGC (the national disaster agency) used
it for communication purposes.
Delays in revising the initial targets set in the HAC and in the Humanitarian Response Plan in all
the three countries reflected the lack of systematic monitoring. This highlights the need for specific
tools to use for this. Lack of systematic monitoring made linking needs and targets challenging, a
situation that was not helped by the lack of outcome monitoring data. UNICEF’s monitoring has
tended to report on accomplishments towards activity and output targets, rather than assessing the
quality of interventions or their outcomes.134 The RTE team partially attributed this to a tendency
of UNICEF staff to view outcomes in terms of behavioural changes, instead of as a sequence of
immediate, medium-term and longer-term outcomes. Most of the limited outcome data collected
by the RTE team, apart from some sections in UNICEF Zimbabwe, came from internal reports
131
An example of a VFM checklist is attached as Annex 8.
132
UNICEF’s Supply Division already has some systems in place. UNICEF could learn useful lessons from the process that
WFP has undertaken during the past couple of years when revising their budget systems to facilitate VFM analysis. The
‘Better Use of Resources’ initiative of the International Rescue Committee provides good practice examples from the health
and nutrition sectors.
133
https://ona.io/services-rapidpro.html. UNICEF’s Innovation Fund supported development of these platforms to enhance
UNICEF’s regular programming.
134
Warner, Alexandra, What is monitoring in humanitarian action? Describing practice and identifying challenges, ALNAP/ODI, 2017.
68
of UNICEF’s partners based on their own post-distribution monitoring systems. These gave an
indication of how assistance had been used and the extent to which it had addressed priority needs.
UNICEF also faced challenges with internal coordination in Mozambique and Malawi, notably
between capitals where the country offices were located and the hubs, which were set up to manage
field operations. Contributing factors included rapid turnover of UNICEF surge staff, gaps in
information management systems and insufficient monitoring visits by some section heads.
Based on findings from KQ 1, KQ2, KQ3, KQ4, KQ5 and KQ6
R9. Strengthen information management and monitoring systems to guide UNICEF
interventions and more clearly demonstrate outcomes, including at the community level.
UNICEF country offices, with support from ESARO
• Capture lessons learned and consider replicating UNICEF Mozambique’s approach of helping
the government better visualize, analyse and use data.
• Set more ambitious targets for regular field monitoring visits by UNICEF staff. These can double
as programme and partner support visits. UNICEF staff must be prepared to monitor in sectors
beyond their own specialty to the extent possible.
• Consider alternative options, including joint monitoring missions, to supplement and verify
partner reporting. This has benefits for cross-sector learning as well as accountability and quality
assurance.
• Adapt humanitarian performance monitoring (HPM) systems to better measure immediate
outcomes based on the CCCs, using data from post-distribution monitoring by UNICEF and
partners, as well as community complaints and feedback mechanisms. This will give a better
understanding of the appropriateness and effectiveness of assistance while assisting in updating
needs assessments. Outcome monitoring systems should be applicable to cash-based and in-kind
distributions and capture relevant cross-cutting data (protection, gender, people with disability) to
the extent that is appropriate to the phase of the response, using a ‘good enough’ approach.
135
• Agree with partners on minimum standards
136
for measuring outcomes and incorporate these
standards into PCAs for implementing partners based on the ‘good enough’ system described
above. Capture lessons learned and consider replicating UNICEF Mozambique’s approach of
supporting government capacity for data visualization, analysis and use.
• For future similar emergencies where a disaster affects more than one country, UNICEF should
establish a working group of monitoring and evaluation focal points as a pilot to support quality
assurance and promote organizational learning.
UNICEF headquarters
• Develop minimum standards and related guidance for measuring outcomes during emergency
response based on a ‘good enough’ approach, including embedding such standards in PCAs.
CONCLUSION 10.
Need to improve preparedness for RTEs
135
‘Good enough’ in this context means choosing a simple solution rather than an elaborate one. ‘Good enough’ does not mean
second best: it means acknowledging that, in an emergency response, adopting a quick and simple approach to outcome
measurement and accountability may be the only practical possibility, and improvements can be made over time. See: Good
Enough Guide: Impact Measurement and Accountability in Emergencies, Oxfam, 2007.
136
These will normally include post-distribution monitoring and periodic knowledge attitudes and practice (KAP) surveys.
69
A challenge faced during this RTE was differing expectations about which performance
benchmarks should be used to measure performance. The TOR made reference to the HAC, the
CCCs, country response strategies and sector strategies. UNICEF country office humanitarian
performance monitoring systems – along with inter-agency Humanitarian Response Plans,
Humanitarian Country Team action plans and cluster strategies – were also key references for this
RTE. The emphasis on timeliness for RTEs and the need to minimize transaction costs rules out a
scoping mission to reach a consensus on suitable benchmarks during the inception phase. A
number of international humanitarian agencies that routinely conduct real time evaluations use
performance measurement templates with a scoring system that is contextualized for a specific
disaster response.137 As part of institutional emergency preparedness, UNICEF could develop a
suitable template based on the revised CCCs, together with guidance on contextualizing indicators
for scores. A more standardized approach to RTEs has many potential advantages, including
streamlining RTE processes, clarifying expectations about how performance is measured and
making it easier to compare different RTEs to help identify institutional gaps, facilitate institutional
learning and reinforce accountability.
Based on findings described in the Methodology section and KQ2
Recommendation targeted at UNICEF’s Evaluation Office
R10. For future RTEs, UNICEF should develop and field test a template
138
and associated
guidelines based on the revised CCCs, using indicators that are developed during the
inception phase that are adapted to the operating context. These should be part of the
toolkit supporting UNICEF’s preparedness for corporate emergencies.
• Commission a review of methodologies and learning processes used by humanitarian agencies
that routinely conduct RTEs to gain an understanding of the advantages and disadvantages of
various options.
• Develop a pilot version of a performance assessment template for RTEs and field test it through
a consultative process. Communication and information strategies should support ownership,
implementation and learning loops.
• Periodically produce a synthesis of results that can support UNICEF’s learning and
accountability.
137
See, for example, CARE International’s Rapid Accountability Review guidelines and examples.
138
The CCC matrix in Annex 5 provides an example of such a template.
