Terms of Reference for Final Evaluation of Social transfers for vulnerable people (SAGAL) project

The number of households reached under the Shock Responsive component2 is 10,111 households.

2.    Purpose of the Final Evaluation Consultancy

The purpose of this endline evaluation is to assess the relevance, effectiveness, efficiency, impact, sustainability, and scalability of the SAGAL project. The evaluation will also identify the best practices, lessons learned, challenges, and recommendations for future programming and policy development. The evaluation will cover the entire project period (202o-2024), all 4 models, the Shock Responsive Component, the system building component, all the project locations and beneficiaries.

The evaluation will put emphasis on analysis on evaluation questions prioritised by the Somali Cash Consortium and MOLSA, to support the final reporting of progress against log frame indicators. The evaluation will seek to respond to four main evaluation questions based on the OECD DAC evaluation criteria.It will thus address the following key questions:

•       To what extent did the project achieve its objectives and expected results?

•       How did the project contribute to the well-being and resilience of vulnerable children and their families?

•       How did the project address the specific needs and rights of different groups of children, such as girls, boys, children with disabilities, etc.?

•       How did the project integrate cross-cutting issues such as gender, disability, child protection, etc.?

•       How did the project adapt to the changing context and needs of the beneficiaries and stakeholders?

•       How efficient and cost-effective was the project in terms of resource utilization and management?

•       How the programme could have achieved better results considering the limited resources available from FGS and FMSs? And provide examples of countries where this was done.

•       How sustainable and scalable are the project outcomes (ST1, ST2, ST3 and ST4) and impacts?

•       How can the Shock Responsive component and the implementation of the H-D-P nexus be improved in terms of government ownership and scale up as part of the anticipatory response framework put in place by SODMA?

•       Given that SAGAL was part of the broader programme ILED, the evaluation should takes into account what ILED set out to do “[…] targets specifically vulnerable households and aims to increase their income and consumption and to improve their ability to cope with shocks through predictable access to transfers, while enhancing and protecting human capital”.

•       What are the main factors that influenced the success or failure of the project?

•       What are the best practices, lessons learned, challenges, and recommendations for future programming and policy development? In particular in the hypothesis the programme was extended:

•       How could the program better contribute to supporting the system building of the social protection sector? What can be learned from its implementation and results achieved?

•       How could the program further improve access to complementary services for social transfer beneficiaries and their community members?

3.       Proposed Assessment Methodology

The evaluation will employ a mixed method study design utilizing quantitative and qualitative methods. In addition to this, desk reviews will be utilized to collect quantitative data from relevant project documents, as well as monitoring data to assess the performance of project indicators. Chiefly, Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs) with program and government staff, social protection sector stakeholders (government, donors, other implementing agencies and technical experts) and beneficiaries will be used to collect additional data needed to complete the evaluation.

When successful, the consultant will be expected to:

•       Carry out a desk-review of relevant project documents including project proposal documents, quarterly reports, annual reports, baseline report, monitoring and evaluation plans and more so the project’s end line survey reports for each model.

•       Develop and present an inception report, detailing design, methodology, tools, work plan and budget to carry out the assignment in consultation with the Somali Cash Consortium and MOLSA

•       The consultant will be expected to develop a sampling design for additional Qualitative Assessments (QA) FGD, In-depth Interviews and key Informant Interviews, as necessary.

Note that the actual end line data collection/survey for each model will have already been done under a separate arrangement. The consultant for this assignment is only required to collect additional qualitative data and utilise the available survey reports to complete the evaluation process.

End line data will be analysed in order to identify and test assumptions of the program design as well as generate both intended and unintended outcomes of the project. The evaluation will review key outcome

indicators listed in the logical framework and explore in what ways the project has contributed to the goal set out at the beginning. The consultant will be expected to strongly involve key MOLSA, Federal Member State (FMS), district government, SCC and implementing partners’ staff and other appropriate stakeholders in the course of the assessment. The expected deliverables of the final evaluation are:

§  Inception report, detailing the methodology for the assignment, and all data collection tools and assessment protocols

§  Data tools in paper and digital formats

§  Cleaned qualitative data stored in an accessible digital format

§  Short, clear preliminary and final evaluation report, documenting process and results, which will be used to assess programme performance over the years of implementation. The report should explicitly discuss the project’s performance against each of the OECD criterion.

§  Slide deck summarizing the main report

§  Updated logframe, clearly showing progress and shortfalls

§  5 Policy/learning briefs (maximum 5 pages each) on key results and learning for each of the 4 models and for the capacity building component (outcome 2)

§  The consultants will be required to produce separate report and brief for Northern Somali regions

§  In person, presentation of the key findings of the final evaluation to key stakeholders in Mogadishu and Hargeisa

The Final evaluation report should follow the following format (maximum of 15 pages, not including annexes):

•       Executive Summary (no more than 2 page)

•       Methodology

•       Key Results/Findings against OECD DAC Criteria for each model (ST1, ST2, ST3 and ST4) and for the capacity building component

•       Conclusions and recommendations for each model (ST1, ST2,ST3,ST4) and for the capacity building component

•       Annexes – to include data collection tools (including any questionnaires), slide decks, learning briefs, coding sheets, etc.

4.      Timeframe

The assessment will take place for a period of 2 months, from December 2023 as proposed below.

Essential

•       Master's degree or higher in social sciences or related fields

•       At least 10 years of experience in conducting evaluations for development and humanitarian projects, preferably in the field of social protection

•       Proven expertise in using mixed-methods approaches and tools for data collection and analysis

•       Knowledge of the context and challenges of Somalia, especially in relation to social protection

•       Excellent communication and writing skills in English and Somali

•       Ability to work independently and meet deadline

•       Previous experience conducting qualitative and quantitative assessments, including sampling, tools development, training of field staff

•       Skills in qualitative and quantitative data analysis using Excel, Epi Info, SPSS or other statistical software package

•       Experience in participatory approaches to data collection

Desirable

•       Understanding and prior experience working in social protection and humanitarian projects in east Africa, especially Somalia

•       Able to implement the research in remote parts of Somalia

•       Have permission to work, travel and undertake primary data collection in the project target locations

•       Have excellent report writing skills

Mandatory Requirements

•       Provide a certified copy of business registration(company/organization

•       Provide a certified copy of tax registration

•       Provide information on ownership structure - name of directors/owners of the company(company/organization)

•       References including names and contact information from previous clients who can be contacted regarding relevant experience

If you are interested and meet the requirements, please submit an application by Sunday, 12th November, 2023 by email to: [email protected] outlining your exact availability in line with the

approximate timelines.

Application submitted after the deadline will not be considered. The subject

of the application should be “Somali Cash Consortium – “SAGAL Final Evaluation”.

The expression of interest should contain: (a) a technical offer and (b) a financial offer, comprising:

A.      Technical offer:

•       Up to date CV of the consultant(s) explaining how you meet the skills and experience required.

•       Technical proposition detailing proposed methodology and proposed sampling approach (max 3 pages). Please ensure this reflects the data collection plan outlined above

•       At least 3 examples of similar work undertaken and references

B.      Financial offer:

•       A list of all expenses expected to be incurred by the consultant(s) including professional daily rates.

•       Costs of transport in-country and accommodation while on field visits in Somalia

Disclaimer:

Only shortlisted candidates will be contacted for further discussions. SCC and MOLSA reserve the right to reject any or all applications without citing any reason. We are committed to safeguarding and promoting the welfare of our beneficiaries, and especially children, and expect all consultants to share the same commitment.