North West Balkans
Languages
Terms of Reference for End of Project Evaluation of the Child Protection – ENVISION III Project
Terms of Reference
for
End of Project Evaluation
of the
Child Protection – ENVISION III Project
Sarajevo, 2023
Save the Children believes every child deserves a future. In Northwest Balkans and around the world, we work daily to give children a healthy start in life, the opportunity to learn, and protection from harm. When crisis strikes, and children are most vulnerable, we are always among the first to respond and the last to leave. We ensure children’s unique needs are met, and their voices are heard. We deliver lasting results for millions of children, including those hardest to reach. We do whatever it takes for children – every day and in times of crisis – to transform their lives and the future we share.
Save the Children in Northwest Balkans (SCiNWB), based in Sarajevo, Bosnia and Herzegovina, conducts its activities in three countries: Bosnia and Herzegovina, the Republic of Serbia, and the Republic of Montenegro, and supports the realization of regional initiatives in Southeast Europe.
3. About CHILD PROTECTION – ENVISION (ENVISION – Creating ENabling enVIronment for Social InclusiON of children – phase III)
Street-involved and at-risk children present one of the most jeopardized groups in the state of social need. The reasons for this lie in the systemic exclusion from all relevant social systems, the lack of opportunities, and the lack of systemic support to combat poverty. Most street-involved children live below the poverty line, often being part of a cycle of poverty that is generationally passed on. As a group, street-involved children are ‘hidden’ from the public eye and the main spheres of political, economic, and social life in Bosnia and Herzegovina. Opening of Drop-in Centres (DiC) for street-involved and at-risk children was, on several occasions, characterized as the only positive street-involved ort provided to this population. Strengthening existing and replicating quality services for street-involved children has become a basis of the Save the Children support, which resulted in established strategic partnerships with governmental and non-governmental institutions in Mostar, Banja Luka, Bijeljina and Brčko in Bosnia and Herzegovina, as well as Podgorica and Nikšić in the Republic of Montenegro.
This service consists of several components: basic needs in terms of food and clothes, shelter to legal assistance, psychological counseling, life-skill, and vocational training, medical care to substance abuse programs, school inclusion, and reintegration to access to welfare benefits, as well as programmes to develop their socio-emotional competencies, strengthen the capacities of their families, and provide them with economic empowerment opportunities. Service is being provided within the premises of the DiC) and through outreach work. The Centre’s work is being further developed, with efforts invested in integrating these services into the social protection system.
3.1 Project Objectives
|
Overall Objective |
Contribute to improving the quality of life of vulnerable girls and boys in Bosnia and Herzegovina and Montenegro in selected locations |
|
Specific objective |
DiCs in selected areas are operational (targeted girls and boys and their families have access to quality care and educational services) and meet preconditions to become self-sustainable |
|
Result 1 |
% of girls and boys who attest improved quality of life in selected locations, disaggregated by gender and age |
|
Result 2 |
% of targeted girls, boys and their families, have access to social and health services, education (through referral system(s)) disaggregated by gender, age and services |
|
Result 3 |
% of targeted girls and boys aged 3-6 have improved their Socio-emotional Competencies disaggregated by gender and age |
Save the Children is seeking the Evaluator - consultant or research consulting firm to design and conduct:
End of Project Evaluation of Child Protection – ENVISION III Project
4.1 Target areas of the Evaluation
End of Project Evaluation will be conducted in the locations where Save the Children’s partners operate, focusing on concrete work in four targeted geographic areas of Bosnia and Herzegovina (Banja Luka, Bijeljina, Brčko, and Mostar) and two geographic areas in the Republic of Montenegro (Nikšić and Podgorica).
4.2 Objectives of the Evaluation
The objectives of this Evaluation are to:
1. Undertake an evaluation to assess the project’s:
Relevance – the extent to which the objectives were in accordance with the beneficiaries’ needs and priorities
Effectiveness – the extent to which the targeted project objectives were achieved
Efficiency – were the resources envisaged for this project converted into results
Sustainability – to what extent and which project activities are likely to continue after the project
Impact – what are the direct, indirect, intended, unintended, positive, and negative long-term effects of the project’s interventions.
2. Provide recommendations for future Save the Children interventions: This evaluation should serve as a learning component for SCNWB to improve its programmatic and project interventions, which could lead to positive change and improved response to the needs of the most vulnerable populations in the region. The evaluation will also serve to inform a broader range of stakeholders in this area, aiming to improve the coordination between various potential changemakers and generate joint efforts.
4.3 Evaluation Questions
The End of Project Evaluation questions should be posed in accordance with the defined Evaluation Objectives:
Relevance
· Was the project design appropriate for the context within which it operated and consistent with beneficiaries’ needs?
· Were vulnerable groups (children and their parents from vulnerable groups) sufficiently well included?
Effectiveness
· What have been the major outcomes/achievements of the project? How were they achieved?
· In what ways have the project’s recipients benefited from project interventions?
· To what extent have the project interventions directly contributed to the identified outcomes?
· To what extent have the external factors contributed to the identified outcomes, if any?
Efficiency
· How well did partner organizations manage their physical and financial resources, and were they in accordance with Save the Children International standards?
·
Sustainability
· Were sustainability and exit strategies sufficiently considered in project design and implementation?
· How likely are each partner organizations to continue the project activities even after the end of the project? What are the differences between each partner organizations’ capacities?
Impact
· If any, how has the project produced long-term effects with its interventions (direct, indirect, intended, unintended, positive, and negative)?
4.4 Methodology
The evaluator should employ a rigorous mixed-methods design, including but not limited to an extensive desk review of the relevant project and secondary documentation, analysis of key informant interviews (KIIs), and focus group discussions (FGDs). Moreover, the evaluation should consist of triangulated data among different sources to cross-validate data and capture different dimensions of all evaluation questions.
The evaluation methodology should consist of:
1. Desk review: Review of all relevant project materials, including but not limited to quarterly and annual reports, external evaluations, project records, etc. Relevant secondary documentation should also be included. The evaluator should assess all progress reports and compare them to the original project description and is particularly welcome to apply content analysis of the documentation. The SCNWB staff will provide all documentation to the evaluator.
2. Mixed methods approach: Data collection tools should be developed in line with the mixed methods approach, participatory and children-friendly evaluation methods appropriate for responding to the evaluation questions. The evaluation findings should be cross-validated through various data sources and analyzed accordingly.
3. Representative sample size: Considering the wide reach of project activities in two different countries and six cities, at least 50 different stakeholders (partner organizations’ staff, SCI staff, relevant governmental or nongovernmental representatives), including a minimum of 5 beneficiaries per targeted project location should be included in this evaluation through KIIs and FGDs.
4. Field visits: The evaluator should visit all partner organizations to gather a more outstanding picture of the interventions and conduct all data collection in person. These visits should also serve as the data quality assessment of partner organizations’ reports.
4.5 Timeline of the Evaluation
The Evaluation is expected to commence on March 27, 2023, and to be completed by the beginning of December 2023. The final report has to be submitted to SCNWB the latest on December 8, 2023.
4.6 Requirements Related to Gender
Evaluation and study design, methodology, data collection, and analyses should capture the situations and experiences of both males and females. The Evaluator should collect appropriate sex-disaggregated data and ensure that an evaluation is gender-sensitive or “engendered.” To engender an evaluation is to view the evaluation process, dynamics, design, and the key elements of each evaluation stage through a “gender lens”.
By engendering an evaluation, all aspects of that evaluation deeply consider gender needs and issues; the evaluation is responsive to the cultural context in which gendered relationships play out; and inquiry is focused on the data and analysis needed to achieve gender equality and empowerment objectives.
Engendering an evaluation means that all stages of the evaluation reflect:
1. An awareness that the degree and meaning of program participation, program results, and potential sustainability are shaped by gender
2. A recognition that explicit attention to gender issues must be integrated into the evaluation
3. A commitment to examining the extent to which gender equality was achieved as a result of the strategy, project, or approach that was implemented.
Responsibilities of the Consultant/Consulting firm:
1. Consultant/Consulting firm is fully responsible for implementation of this assignment, including methodology design, data collection, data analysis and reporting
2. Consultant / Consulting firm will liaise with Save the Children staff throughout the process, providing regular updates and seeking their input and advice where necessary
3. Consulting firm will prepare a detailed task timeline and draft methodology (inception report) and work with Save the Children to finalize the methodology
4. Consulting firm is responsible for methodology design, including sampling and tools for data collection
5. Consultant/Consulting firm is responsible for conducting data analysis based on collected data through evaluation
6. Consulting firm will adhere to the timeframe of the consultancy
7. Consulting firm will deliver periodical and final report in the English language
8. All the data collected during the research/study period should be protected in line with provisions of the General Data Protection Regulation 2016/679, and national regulations on the protection of personal data in Bosnia and Herzegovina and Republic of Montenegro
9. Consulting firm will ensure the ethical participation of children, and ensure full implementation of Save the Children's policy on Child Safeguarding.
Responsibilities of Save the Children:
1. Providing all background materials for the research consulting firm
2. Ensuring the participation and support of implementing partners and beneficiaries
3. Monitoring adherence to specified deadlines; facilitating access to required information
4. Providing guidance throughout all phases of execution, approving all deliverables, and facilitating, access to any documentation (or any person) deemed relevant to the process
5. Share deliverables with stakeholders
6. Perform quality control, as required for all deliverables.
6. Deliverables and reporting requirements
The consultant/consulting firm should submit the following deliverables during the evaluation process:
1. Draft methodology for the Evaluation (including evaluation design)
2. Inception report (methodology, sampling, tools for data collection, approach to data analysis, and details of Evaluation Work Plan (draft implementation schedule)
3. Draft and final report of Evaluation submitted in English; the final report will incorporate Save the Children’s feedback into the draft report
4. Datasets – quantitative data/datasets created or obtained in the performance of this task; datasets) must be delivered in a non-proprietary, machine-readable format; the dataset must be organized and documented for use by those not fully familiar with the intervention or evaluation.
7. Content of the evaluation report
Save the Children expects that the final report will include the following sections, at a minimum:
1. Cover Page
2. Acronym List
3. Executive Summary
4. Project Background
5. Purpose and Scope of the Evaluation
6. Evaluation Questions
7. Evaluation Methodology
8. Evaluation Findings
9. Evaluation Conclusions (successes, challenges, and lessons learned)
10. Recommendations
11. Attachments (evaluation SoW/ToR, data collection instruments, photos, charts, graphs, additional analyses)
12. Bibliography (APA Style of referencing).
The final report should not have more than 50 pages (without attachments).
8. Evaluation schedule
The below evaluation schedule can be modified and adjusted according to the Project’s overall implementation plan. In order to perform all scheduled tasks, Consultant/Consulting firm shall be hired for the period March 27, 2023 – December 15, 2023. The Consultant/Consulting firm shall prepare and implement the work plan in consultation with and support by the relevant personnel of Save the Children and partner organizations, in accordance with the following dynamics:
|
Activity |
Time-frame |
|
Draft methodology for the Evaluation (including evaluation design) |
Until April 7, 2023 |
|
Drafting of the work plan and preparation of detailed work methodology and submitting the Inception report with a detailed work plan and methodology for the evaluation |
Until April 17, 2023 |
|
Data collection (Desk review, KIIs, FGDs) and analyses of collected data |
From April 17 to October 10, 2023 |
|
End of Project Evaluation – draft report |
Until November 20, 2023 |
|
End of Project Evaluation – final report |
Until December 8, 2023 |
|
Total estimated number of consultancy days |
40 |
9. Selection criteria
Save the Children will engage an experienced, independent third-party Consultant/Consulting firm to conduct this evaluation. The third-party evaluator will:
1. Be financially and legally separate from Save the Children and partner organizations
2. Have staff with demonstrated knowledge, skills, and experience in conducting evaluations of child protection interventions
3. Provide an outline for the evaluation, major tasks, and schedules prior to initiating the evaluation taking into account the following guidelines:
a. Methodological approach
b. Budget
c. Curriculum Vitae with a detailed summary of the evaluation of programs/ projects conducted previously for individuals/companies. If the evaluation was done for a company/organization, please include a profile of such company/organization
d. A minimum of two letters of reference from organizations with which it/s/he has conducted previous consulting work
e. A minimum of three examples of evaluation reports that the consultant/consulting firm has led.
Save the Children will make the award to the Bidder whose proposal provides the best value, considering both technical and cost factors. Technical and cost factors will be evaluated relative to each other, as described herein. The technical evaluation factors, taken as a whole, are of greater importance than cost or price in determining the best value.
Bidders should note these criteria:
(1) serve as the standard against which all proposals will be evaluated, and
(2) serve to identify the significant matters which Bidders should address in their proposals.
The proposal submitted will be the primary document upon which each Bidder will be evaluated.
Save the Children reserves the right to waive any minor or technical defects or irregularities and reserves the right to reject any or all bids.
Save the Children emphasizes that all the accompanying costs of the process of evaluation and implementation of the engagement (travel costs, accommodation, etc.) should be included in the offer.
Candidates interested in the engagement should send their bids to Procurement.NWB@savethechildren.org no later than March 20, 2023. Only short-listed candidates will be contacted.
Please find enclosed link with specific documents for application: https://drive.google.com/file/d/1r3SpctF7ssDMTjRg3OpdDlAGjqSfZH98/view?usp=share_link
Special requirement:
No person working on any task for or on behalf of Save the Children may in no way whatsoever be involved or related to child abuse or exploitation as defined in Save the Children's Child Safeguarding Policy.
The contracting authority also requires that the personal data of all evaluation participants must be protected and stored in accordance with the General Data Protection Regulation - (EU) 2016/679 and the national law on personal data protection in Bosnia and Herzegovina (BiH Law on Personal Data Protection, Official Gazette of Bosnia and Herzegovina No. 49/06)
