2.0 PROJECT DESCRIPTION

 World Vision and four local partners are implementing Multisector Emergency Response Project (MERP) in Somalia and Somaliland with funding support from the United States Agency for International Development (USAID) through the Bureau for Humanitarian Assistance (BHA).  The Partner—African Relief and Development (ARD), Centre for Research and Integrated Development (CeRID), Mandher Relief and Development Organization (MARDO), and Somali Relief and Development Action (SRDA) work in difficult to access areas in South West State (Bakool, Bay) and and Jubaland (Luuq district of Gedo region), while WV is a direct implementer in Somaliland (Woqooyi Galbeed and Togdheer regions) and Doolow district of Jubaland State. The action is also undertaken together with selected governmental line ministries (Ministry of Health and Ministry of Agriculture), local authorities, community level partners and additional relevant stakeholders across the project districts. The goal of project is to restore the wellbeing of vulnerable communities in Somalia affected by the aftermath of severe flooding, drought, and locusts affected communities; including IDPs, and host communities in Somalia and Somaliland. The number of people targeted is 556,454. Of those, 223,197 is internally displaced persons affected by recurrent drought and insecurity.

WV proposes a performance evaluation that will measure the impact and effectiveness of the program in the target locations with the communities and the provincial authorities. WV also proposes a performance evaluation to ensure that the project met its intended goals and outcomes and how it can be strengthened in the future. The study will be carried out externally through a participatory approach involving partner organizations, line ministries, and beneficiary communities. The evaluation results will help the key stakeholders measure the level of project success concerning service delivery to the project beneficiaries.  More importantly, this end of project evaluation will generate recommendations and lessons learnt for ongoing and upcoming World Vision projects.  

2.1 Geographic Scope and Target Population

The project targets 11 districts including Berbera and Gabiley districts of the Woqooyi Galbeed region; Burco and Odweyne districts of the Togdheer region; Wajid and Huddur districts of the Bakool region; Luuq and Doolow districts in the Gedo region; and Qansaxdhere and Berdale districts of the Bay region in Somalia. And evaluation will take place in all the targeted districts. In addition, this survey will assess progress made from the inception and the overall achievement of project outcomes.

2.2 Context

This   evaluation is to be done in relation to the context taking into consideration the changes that took place during the project implementation period (Covid19, political tension/election in Somalia, droughts, flooding, and insecurity). The survey team will also analyze on how the changes in context affect project implementation and impact including the effectiveness of adaptation efforts to be made by the implementing partners, communities and other stakeholders.

2.3 Community Level Partners

The Project end-line evaluation will involve all stakeholders who were involved in the project design and expected to engage into implementation such as:

• Community groups and committees to be involved in project implementation (WASH, H&N and Protection, FSL).
• Implementing partners-African Relief and Development (ARD), Centre for Research and Integrated Development (CeRID), Mandher Relief and Development Organization (MARDO), and Somali Relief and Development Action (SRDA)
• The relevant line ministries (Ministry of Health, Ministry of Gender and Social Affairs, Ministry of Water and Mineral Resources, Ministry of Agriculture and Livestock)
• Local Administration offices in the project implementation districts

3. METHODOLOGY

This section provides a detailed description of the end-line evaluation process. It focuses on the evaluation design, data collection and analysis and reporting process.

3.1. Scope of the Evaluation

The end of project evaluation will provide a detailed analysis of achievements against the anticipated objectives (baseline values) and measure up-to-date performance of the project. This aims to showcase the project’s relevance of the interventions, efficiency, and effectiveness impacts, of the project intervention towards future sustainability.

3.2 Evaluation Type

This end-project evaluation aimed at assessing the impact and effectiveness of the program in the target locations with the communities and the local authorities. WV also proposes a performance evaluation to ensure that the project met its intended goals of restoring the wellbeing of vulnerable communities in Somalia affected by the aftermath of severe flooding, drought, and locusts-affected communities; including IDPs, hosting communities in Somalia and Somaliland and sectoral outcomes and how it can be strengthened in the future.  World Vision Somalia will hire an external consultant through a competitive process to carry out a quality project end evaluation.

3.3 Evaluation Purpose and Objectives

World Vision Somalia will conduct this evaluation of the project to identify the impact of the project and to assess the relevance of the project logic, and effectiveness of the project models to realize the project outcomes. It further aims to assess the effectiveness of the integration model of the BHA project. World Vision would also like the evaluation to assess the appropriateness of the approaches used in the program, as well as the potential sustainability markers that have been achieved through the project’s implementation. The end-line evaluation will also help to draw key lessons learnt and the best practices to the project stakeholders.

 In particular, the project evaluation will be shaped by the following specific evaluation objectives; 

The objectives of the evaluation are to:

1. Establish the extent to which the project achieved its outcomes among target populations (specifically vulnerable groups including, PLWs, children under 5, women-headed households) and determine the impact or potential impact of the project;
2. Determine how appropriate the project (and the approaches used) was in addressing the root cause and responding to the needs of the community;
3. Document sustainability practices (mechanisms, plans, structure) and identify key sustainability recommendations which can be used to inform future resilience programs, and
4. Highlight best practices, promising approaches and ways to improve programming;
5. Assess the strengths and challenges of the project
6. To determine how the project involved and benefited the community throughout the planning, design, implementation and, monitoring processes.
7. Asses the overall management and structure of the project
8. Assess the strengths, areas of growth and challenges of the project

1. Provide specific and practical recommendations and document lessons to be utilized for the future projects
2. Assess adequacy and efficiency of planning, monitoring and reporting systems, and use of both material and financial resources

3.4 Evaluation Questions

1. To what extent have the activity’s interventions adhered to planned implementation - schedules, participant targeting, resource transfer composition/quantities, inputs and service delivery, and outputs - and achieved intended goals, purposes and outcomes? 
2. Did interventions reach the appropriate target groups and individuals within the target areas? What factors promoted or inhibited adherence to plans and targets?  To what extent has the intervention appropriately assisted the affected population? How has management adapted the project design or implementation based on monitoring information and feedback from the target population?  
3. What were the main themes of changes happened as result of the project intervention?
4. What sustainability drivers did the project engage?
   1. How appropriate are these drivers for moving towards sustainability in the project?
   2. What other drivers or mechanisms could programs consider for future programming?
5. What practices and approaches should be highlighted and considered for scaling?
6. Provide specific and practical recommendations and document lessons to be utilized for future interventions
7. How did these affect the project implementation and how were they handled?
   1. Identify opportunities for the project if any
   2. and how they were exploited/ neglected
8. How equitably has the project benefited the; women, men, boys and girls, IDPs and host community?
9. What mechanisms have been put in place in order to sustain the key activities, outputs and outcomes that were initiated by the project activities?
10. How did these affect the project implementation and how were they handled?
11. Identify opportunities for the project if any and how they were exploited/ neglected?
12. The best practices that have been adopted and how can they be replicated in future projects or partners for adoption?

3.5 Study Design

The project will employ a mixed-methods design for the final evaluation. A quantitative survey will be used to produce indicator values and collect demographic information about the beneficiaries. A qualitative component using Key Informant Interviews (KIIs), Focus Group Discussions (FGDs) and observation will provide insights into the context and help to explain the reasons for the changes observed

3.6 Data Collection Methods

3.6.1 Quantitative Data Collection

The quantitative Population-Based Survey will be administered among a probability sample of participants and households in the target areas. The sample size, sampling frame, and data collection tools will be designed so that data can be statistically compared to test for difference. The questionnaire will be designed and administered using Kobo Toolbox/FieldTask.

3.6.2 Sampling Design

The evaluation will adopt the two-stage cluster sampling design where the target population will first be divided into clusters; in this case the clusters will comprise of the villages/communities/groups in the target areas.  These clusters are randomly selected from a list of all clusters. In the second stage, a sample including households in these communities is randomly selected from each of the sampled clusters using the probability proportional to size (PPS) method.

3.6.3 Sampling Frame

To develop sampling frames, the project will consider its targeting strategy and the different target populations for various interventions being a Multi-Sector response. The survey design will require multiple sampling frames to organize the target groups that will receive a similar set of interventions. The sampling frame for the Population-Based Survey will be developed based on census data and community records to appropriately reflect the target population. The sampling frame for the Participant Based Survey will be developed based on the participant register to appropriately reflect the target population. The sampling unit is the household, where a knowledgeable adult will be answering the questions. The sampling frame will include the following key elements collected at registration:

• Unique household identification number
• Household contact information (including name, physical location, primary and secondary phone number)
• Household characteristics (gender composition, size, primary and secondary livelihood activities)
• Intervention(s) to be received/or received
• Participant target criteria met

3.6.4 Sample Size Calculation

To derive the sample size, the key the purpose of the evaluation (comparative), the key indicators of interest, and the sampling methodology; in this case two-stage cluster sampling will be considered. For the evaluation survey the appropriate sample size for comparing the values of indicators collected at two points in time: at the start of the activity and after the activity is completed will be calculated using a two-stage cluster sample design. The sample size will be estimated for each key indicator and adjusted based on the proportion of households in the sub-set.

Using Feed, the Future Sample Size Calculator, WV calculated sample sizes for the key project indicators for comparing the values at two points and found that the sample size was largest using Food Consumption Score (FCS). Additionally, these calculations meet the minimum number of respondents recommended by BHA for indicators expressed as a proportion using Simple Random Sampling, Systematic Random Sampling and PPS, which is 388 and 776 respondents.

BHA Multi Sectoral Emergencey Project Final Evaluation ToR

Qualifications

The consultant must have proven expertise and experience in social research with a special focus on Agriculture, Agricultural Economics, Health and Nutrition, Development studies, Baseline, end-of-project evaluations, midterm evaluations, and impact assessments and be able to implement the final project evaluations in Somalia following the required procedures. Proof of these is to be provided by submitting, together with the application:

·         An overview of relevant works

·         Working samples

·         Contact details for references

·         The proposed consultant’s/research team's CVs

 Requirements in detail:

·         In-depth knowledge of Somalia and its regions including government and community-level service delivery structures. Understanding of the local context, political and security environment. The CVs of the consultants with extensive working experience or studies conducted in the Horn of Africa, especially Somalia/Somaliland is an added advantage. CVs, assignments, and contracts will be reviewed to check consultants' suitability for the end-project evaluation.

·         The lead consultant should have a Master/Ph.D. degree in Agriculture, Agricultural Economics, Health and Nutrition, Development studies, Peace and security, Monitoring and Evaluation, International studies and 10-15 years of consultancy experience. Interested consultants and firms have to submit Reports and previous contracts to ascertain the suitability of the consultants with the evaluation at hand.  

·         Has technically sound experience in end-of-project evaluation, baselines, and Studies in the Horn of Africa or Somalia.  Consultants are expected to submit reports of the past end project evaluation, baselines, and assessments along with their application package. 

·         Has extensive experience in multi-sectors including Emergency programming, Food Security, livelihoods, Cash transfers programs, and Health and Nutrition. Consultants should able to provide a good track record of their past experience in evaluating cash transfer programs, WASH, and Health and Nutrition in their technical proposals.  Relevant reports and contracts to prove the consultant’s experience in specified thematic areas should be submitted along with the application. 

·         Strong written, communication, and interpersonal skills in English with substantial experiences in training and managing multicultural teams.

·         Proven skills in research, monitoring, and evaluation

·         Proven experience in conducting qualitative, quantitative, and mixed methods evaluation studies. Consultants will be needed to submit evaluation, assessment and baseline reports with strong quantitative and qualitative methodologies.

·         Computer proficiency with excellent MS Office knowledge (Word, Excel, PowerPoint), SPSS, and STATA. Submitted reports by consultants will be reviewed to check if the consultants have excellent knowledge of data analysis software. 

·         Experienced in undertaking baselines/evaluations in the Horn of Africa any country will be an added advantage, especially Somalia and Somaliland. Consultants are required to submit evaluation and Baseline reports done in Somalia or the Horn of Africa region.

·         Excellent analytical and report-writing skills.

·         Excellent written and spoken English.

·         Excellent time management skills.

·         Ability to work well both independently and in a team.

·         Excellent conflict-sensitive approaches and ability to work in highly sensitive environments

 Proposal Contents

Proposals from Consultants should include the following information (at a minimum)

·         Technical Proposal with clear methodology, including types of Monitoring & Verification tools and analysis

·         CVs of key consultant(s) attached to the technical proposal

·         Proposed timeline/Work plan

·         At least 3 References including names and contact information (at least three similar assignments in a similar context are also required)

     I.        Financial Evaluation

·         A financial proposal with a detailed breakdown of costs (which shall include professional fees and operational budget) quoted in USD. The applicable tax amount must be clearly stipulated and separated from the base costs.

·         Payment Terms

·         Credit Period

4.    Clarification of Bidding Document

A prospective bidder making an inquiry relating to the tender document may notify WVS in writing at [email protected] or [email protected] . WVS will only respond to requests for clarification received no later than 13/03/2023.

5.    Bids Submission

All interested bidders are requested to submit their proposal in English and by email to [email protected]  on or before 20th March 2023.

Proposals should be submitted in three distinct/separate attachments, namely.

1. Mandatory Requirements
2. Technical Proposal
3. Financial Proposal

(Bidders who will combine both technical and financial proposals shall be disqualified)

EMAIL TITLE SHOULD BE; - END OF PROJECT EVALUATION- MULTISECTOR EMERGENCY RESPONSE PROJECT

Bids received after deadline shall not be considered.