Terms of Reference for Project Endline Evaluation Assessment for Kismayo Health and Nutrition Project (KHNP) 2023 – 2024, CARE

Background

Somalia continues to grapple with one of the most complex and enduring humanitarian crises worldwide. Successive episodes of severe drought and flooding, escalating conflict, the worst locust infestation in decades, and the emergence of the COVID-19 pandemic have contributed to unprecedented humanitarian requirements. The ongoing pattern of increased displacement further compounds the difficulties, placing an extra strain on the approximately 3.9 million individuals already enduring prolonged displacement.

In Jubaland state, especially in Lower Juba, there has been an influx of internally displaced persons (IDPs) from neighboring regions, notably southwest state, and surrounding districts, driven by violent conflict and severe drought and flooding. This influx has intensified competition for essential services among the affected population and host communities, exacerbating living conditions. The ongoing crises in Somalia as occasioned by successive rain failures, erratic rains, floods, and conflicts have caused an increase in displacement, malnutrition, and preventable diseases such as Cholera/Acute Watery Diarrhea (AWD). Recent clashes and El Nino related flooding have forced thousands of women and children to seek food and medical support in Kismayo and other few secure towns. 

According to the country-wide analysis of Acute Malnutrition under the Integrated Food Security Phase Classification (IPC) conducted in August 2023 Somalia is facing a severe burden of acute malnutrition, with an estimated 1.7 million children affected, including 430,000 severely malnourished and 1.23M moderately malnourished children, expected in 2024.

Approximately 4 million people across Somalia are experiencing high levels of acute food insecurity (IPC Phase 3 or above) between January and March 2024.  Of these, 3.2 million people (17 percent of the population) are in IPC AFI Phase 3 (Crisis), and around 800,000 people (4 percent of the population) are experiencing worse conditions in IPC AFI Phase 4 (Emergency).

According to the Humanitarian Response Plan (HRP) for 2024, critical nutrition services are needed, including the treatment of children with severe acute malnutrition (SAM), supplementary feeding for children with moderate acute malnutrition (MAM), and pregnant and lactating women, along with micronutrient supplementation and interventions for infant and young children feeding. Additionally, ongoing conflict and the resulting loss of livelihoods due to recent Deyr (Oct-Dec 2023) flooding continues to impact food security in the Lower Juba region, particularly in Kismayo and Afmadow Districts.

In Jubaland State, 233,240 children under the age of five are grappling with acute malnutrition, with nearly 59,380 at risk of severe malnourishment. Stunting among children under 5 in Jubaland state is prevalent at 28.1%. Malnutrition rates among these children are 33.7%, with a 16.3% prevalence of wasting. Poor infant and young child feeding practices are evident, including low dietary diversity (5%), sub-optimal breastfeeding rates (76%), and a limited number of children being breastfed up to 2 years (30%).(JHDS, 2021) The World Food Programme (WFP) identifies Gedo and Lower Juba, regions within Jubbaland, as having the highest levels of insufficient food consumption. In Gedo,22% of the population, and in Lower Juba, 19%, are facing Crisis (IPC 3) and Emergency (IPC 4) food consumption gaps between January and March 2024 (IPC Somalia, 2024). Failed cereal harvests and global price shocks due to the Ukraine conflict have pushed staple and imported food prices beyond the means of millions of households already spending 60-80% of their income on food.

In addition to the Health and Nutrition challenges, internally displaced women and girls in Lower Juba face heightened risk of different forms of gender-based violence (GBV) including domestic violence, sexual violence and exploitation, female genital mutilation (FGM), and early or forced marriage.

CARE is a humanitarian non-governmental organization committed to working with poor women, men, boys, girls, communities, and institutions to significantly impact the underlying causes of poverty. In Jubaland, CARE programs are providing life-saving support to women and children through the provision of Healthcare, Nutrition, Protection, Sanitation and safe drinking water for both Human and livestock use, and Food security/ livelihood support (mainly Cash and Voucher Assistance).

Through MOFA Luxembourg funding projects, CARE aims to save lives, alleviate suffering, and maintain human dignity through gender-responsive health, nutrition and protection interventions for crisis-affected women, men, boys, and girls in Somalia.  The Kismayo Health and Nutrition Project (KHNP) for integrated health and nutrition, in Kismayo, Lower Juba Region, Somalia, is a 1-year CARE in Luxemburg / MOFA -funded project running from 1st June 2023 to 30th July 2024. The project is implemented by CARE, which plays an overall leadership role and implements field activities. This project is in line with CARE’s overall strategic objective for Somalia which includes a long-term commitment to communities and integrated programming where projects contribute to much wider programmatic objectives aimed at addressing the underlying causes of poor health and malnutrition, as well as addressing the new emerging humanitarian crises. This project responds to the grave humanitarian situation among crisis -affected people in Kismayo District, targeting the Internally Displaced Persons (IDP), returnees, and vulnerable host communities by providing multisectoral interventions in Health, Nutritional, and Protection intervention support for acutely malnourished children and pregnant and lactating women.

The interventions provide basic emergency health and nutrition, and protection services including treatment of severe and moderate acute malnutrition, health and hygiene promotion, and protection support to children under 5 and PLWs, and support social behaviors change (SBC) on Infant and young child feeding in an emergency (IYCF-E) practice. Other interventions are the treatment for common communicable and non-communicable ailments Immunization, micronutrient supplementation, antenatal care, skilled birth attendance, postnatal care and integrated risk communication & community engagement (RCCE) and referral services for common communicable diseases.  A Gender Desk facilitates access to safe, confidential survivor centred medical and non-medical GBV services.

Life-saving nutrition services through the CMAM approach are core to the project’s design, the components of CMAM being community outreach and mobilization; management of SAM without medical complications through outpatient therapeutic programs (OTP); inpatient management of SAM with medical complications at stabilization centres (SC); and outpatient targeted supplementary feeding programs (TSFP) to manage MAM. The project teams work through community-based approaches to raise awareness of good nutrition practices (particularly infant and young child feeding (IYCF)) and early referral to nutrition services, given their importance for the prevention of malnutrition, and reducing risk of associated mortality and adverse developmental outcomes.

The ongoing interventions seek to meet the immediate needs of the most vulnerable girls, boys, and women in the new Kismayo IDP in Kismayo town. The Project will prioritize the management of Acute Malnutrition (MAM) and Infant and Young Child Feeding (IYCF) practices. The project targets reaching the following with lifesaving assistance:

·         Treatment of communicable and non-communicable diseases to 14,900 individuals: 8,000 women, 5,796 men and 1,104 children.

·         1,500 mothers with antenatal care.

·         1,500 children under 5 years will be fully immunized against vaccine-preventable diseases.

·         1,300 children aged 6-59 months (500 SAM & 800 MAM) and 500 pregnant and lactating women (PLW) with malnutrition.

·         Target 10,000 Mothers and caregivers with maternal infant and young child nutrition behavior change messaging.

·         Disseminate information on protection to 10,000 persons getting services in the health facility.

·         Provide clinical and psychosocial support to 100 GBV survivors.

Project objective: To improve Health and Nutrition status amongst children under 5 and pregnant and lactating women through the provision of quality Health and Nutrition services and contribute to the reduction of maternal and childhood morbidity and mortality.

Objectives of the Project Endline Evaluation 

The primary aim of the endline evaluation is to evaluate and establish dependable concluding data regarding the project's performance through predefined parameters across its three components: Health and Nutrition, and Protection. The evaluation report will pinpoint best practices and essential insights in both technical aspects and the program management approach, facilitate continuous learning and enhance the effectiveness of humanitarian emergency response. The outcomes of this concluding assessment will be incorporated in future programs and will also be communicated to donor CARE Luxemburg and Luxembourg MoFA Specific objectives include:

·         To determine whether the implementation strategy has contributed to addressing the issues proposed in the intervention of the project (Kismayo health & nutrition project)

·         Assess the results achieved and any discrepancies between planned and achieved objectives (effectiveness);

·         Compare the results achieved with material, financial, human resources, etc. used (efficiency); 

·         Assess the strengths and weaknesses of the overall project implementation methodology as well as the opportunities and constraints for its scale-up. 

·         Assess the quality of the involvement and participation of the various stakeholders (CMP, country office, and project participants) in the implementation of the project.

Evaluation Questions

1.     What were the project's achievements in relation to the project proposal Performance Measurement Framework at outcome levels (immediate, intermediate, and ultimate)?

2.     To what extent does the project comply with OECD-DAC criteria on effectiveness and impact?

3.     What were the lessons learned in terms of the project's contribution towards gender-responsive nutrition outcomes and nutrition services?

4.     What was the impact of project intervention on women, IDPs, returnees, and people with disabilities who received direct support through the project?

5.     What are the key lessons learned and best practices from the program implementation, operating context, and response type as well?

6.     How did the project ensure accountability to the affected population (AAP)? How is community feedback integrated into project activities?

Evaluation Criteria

The evaluation will be done in line with OECD-DAC recommended criteria based on the three criteria – effectiveness, sustainability, and impact. The end-line assessment will contribute to both strengthening the accountability of CARE International in Somalia to its donors and key stakeholders including beneficiaries, and to learn from this experience to inform future programming.

Effectiveness

·      To what extent did the project provide a timely response to the needs of the beneficiaries?

·      To what extent has the project enhanced and addressed the practices related to feeding infants and young children’s practices and GBV through the attainment of knowledge, attitude, and practice changes?

·      What were the key drivers and barriers that impacted the delivery of the project outputs?

·      To what extent has the project used learning to improve delivery?

·      To what extent has the program achieved its intended health and nutrition outcomes?

·      Can changes in health and nutrition outcomes be directly attributed to the program?

Efficiency

·         To what extent has the project been efficient in using resources (funds, human resources, time, expertise, etc.) to achieve the relevant health, nutrition and protection outputs and outcomes?

·         To what extent have project funds and activities been delivered in a timely and cost-effective manner?

·         Was the project management, coordination and monitoring efficient and appropriate?

Sustainability

·      What is the level of community ownership, as reflected in the participation of the community in the project-supported Health facility’s community-based structures?

·      Do the community have the capacity to maintain the benefits from the project with support from CARE or any other donor? If yes, how? If not, what will be the principal challenges in sustaining the program benefits?

·      To what extent has the program achieved its intended health and nutrition outcomes?

·      Can changes in health and nutrition outcomes be directly attributed to the program?

Impact

·      How has the project contributed to the improvement in the quality of life of the people in affected districts?

·      Have there been any other changes (positive and negative) at the household or community level because of the project?

·         How has the health and nutrition status of the target population changed over the course of the intervention or program?

·         What are the observed improvements or deteriorations in key health & nutrition indicators?

·         Has the project affected people in ways that were not originally intended? If yes, what has changed?

·         What was the role of the program partners in delivering the assistance?

·         What is the gender impact of the project in the areas of intervention?

Equity

·      Did the project contribute to equitable participation and benefits to various groups (men, women, boys and girls, and differently abled people)? 

·         Did the project consider gender equality, as well as social inclusion and human rights, environmental protection and climate change (if applicable and where relevant)?

·      Over the past 1 year, were there any undertakings by the project to promote equity? If yes, how? If not, why not?

Approach and Methodology

The final evaluation serves as a performance assessment, utilizing a non-experimental design to enable a simple pre-post comparison of results. This evaluation employs a combination of quantitative and qualitative methods, involving a beneficiary household survey, document reviews, and interviews with both participants and stakeholders. The reporting of project accomplishments linked to specific indicator values will take a comparative analysis approach.

The technical support team is anticipated to put forth a fitting and comprehensive evaluation methodology that includes the right evaluation approach, sampling techniques, data analysis methods, presentation and reporting strategies, and ethical considerations for the evaluation.

Scope of the Evaluation

The end-line assessment will cover the Bacadweyn health facility in Kismayo district of Lower Juba, Somalia. The participants of the evaluation survey will include IDPs, returnees, host rural and urban communities in the target area as well as government authorities and partner agencies.

Responsibilities of CARE     

·         CARE will provide all logistics support including flights, accommodation, transport, enumerators recruitment, payment of enumerators, etc.

·         Pay the consultancy fee for the study.

·         Introduce the consultant to relevant stakeholders.

·         Review and approve the study instruments/tools including the digital quantitative (Kobo, ONA) tool and qualitative guides.

·         Support the consultant in field data collection while supervising the entire process.

·         Overall data quality control and technical review of the final report and approval of the final report

Responsibilities of the Consultant

·         Develop appropriate methodology and tools for the data collection and analysis process. The tools will be shared and approved by CARE before starting fieldwork.

·         Develop a practical work plan for the work.

·         Undertake a desk-based review and lead field-level data collection.

·         Train enumerators who will be involved in the data collection.

·         Conduct data collection including pre-testing of tools in the district covered by the project,

·         Ensure that datasets are clean, complete, and consistent.

·         Analyze data and write baseline survey report.

·         The consultant shall be liable for ethical procedures including obtaining informed consent from respondents.

·         Debrief CARE Somalia/Somaliland after completing the assignment and validate the findings prior to submitting the final report.

·         Prepare and submit a final evaluation report to CARE Somalia/Somaliland as per the time frame set in the ToR.

Key Deliverables 

The evaluation deliverables are:

    I.       Inception Report:  </

Qualification of the Consultant

The evaluation assessment will be carried out by an external party. The consultant or consulting team must have considerable experience in conducting evaluations for projects similar to those supported by MOFA Luxembourg in Somalia-funded projects. Moreover, they should exhibit significant proficiency in the Somali context, with a specific focus on the regions of Jubaland State, to ensure a comprehensive grasp of the local circumstances.

Relevant qualifications for the consultant or team are outlined below.

·         The lead consultant & co-consultant must have the following qualifications: Masters or degree in health sciences, public health, social sciences, sociology, development studies and other health related fields. 

·         Over 5 years of international humanitarian and development experience, specifically in the Somalia context.

·         Demonstrated experience in conducting humanitarian evaluations. Specifically, profound understanding of the factors influencing malnutrition and poor health in the Somalia context, coupled with experience in evaluating health and nutrition programs.

·         Over 7 years’ experience of conducting baselines and endline evaluations on health-related projects in Somalia: especially southern Somalia with International organizations.

·         Substantial experience and understanding of integrated nutrition programming in emergency situations notably Health, WASH (Water, Sanitation, and Hygiene), Protection, Food Security, and Livelihoods.

·         Comprehensive knowledge and experience of the Somalia context, with preference for teams that include Somali speakers.

·         Robust experience and understanding of the application of humanitarian standards such as SPHERE and Core Humanitarian Standards (CHS).

·         Capability to grasp the broader context and provide practical recommendations for implementation and on-the-ground improvements.

·         Proficiency in collecting, analyzing consolidating, and communicating succinctly credible project information.

·         Excellent interpersonal skills.

·         Experience and ability to operate in insecure-volatile environments.

·         Willingness to travel to Somalia within the security framework established by CARE International.

·         Proficient in participatory facilitation and workshop design.

Excellent English report-writing skills

All applications MUST be accompanied by a technical and financial proposal including a brief outline of the proposed methodology, 3 references with contacts, and a tentative work plan as well as the candidate’s availability during the months of May to June 2024. The proposals shouldn’t exceed 5 pages. The deadline for submission of technical and financial proposals is 25th May 2024.

Interested consultants or firms are expected to conduct their own security and accessibility of the target project locations before application. Updated CVs of individuals or profiles of applying companies expressing interest in conducting the assessment should be sent to [email protected]  no later than 25th May 2024. Please indicate “Kismayo Health and Nutrition Project (KHNP)– Project End-line Evaluation’.