1. Purpose of the Evaluation / Review 

The present Terms of Reference (ToR) provide the framework for the evaluation of a Vocational Education and Training program in the Great Lakes region, supported by Switzerland since 2012. The evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the next phase program design. The evaluation shall be used to assess the implementation of activities leading to results and consider adjustments. The evaluation report’s purpose is to:

                      I.        Enhance the program’s relevance, efficiency, sustainability and effectiveness;

                     II.        Suggest improvements for the following phase III;

                   III.        Demonstrate the program’s achievements, challenges and lessons learned including governance and gender dimension;

                   IV.        Generate findings regarding the effectiveness of the Swiss support to local Vocational training centers including its effects on women and girls;

                    V.        Identify any intended or unintended outcomes which may already be visible; and

                   VI.        Sketch out realistic and actionable lessons learned and recommendations for the planning of the next phase of the Program;

                  VII.        Indicate any risks that may compromise the successful implementation of the program and suggest actions to overcome it.

2.  The Evaluation/Review context

In 2012, the Government of Switzerland, acting through the Swiss agency for Development and Cooperation (SDC) established a 12-year TVET program in the Great Lakes’ Region aiming at supporting the Governments of Rwanda, Burundi and DRC (Province of South Kivu) to improve access, quality and relevance of their respective TVET systems in order to address a key development challenge consisting of limited employability of their human capital due to low quantity and quality of skills produced by the TVET sub-sector. Limited to Rwanda during the first phase (November 2012 - December 2015), the PROMOST program was extended to DRC and, Burundi during the second phase (February 2016 – January 2019). The Swiss Foundation for Technical Cooperation (Swisscontact) has been mandated by SDC to implement the first two phases of the program in the region. The program’s overall goal is to contribute to increase employment and income generation for the rural youth, men and women in targeted geographical areas by improving their access to quality and labour market oriented vocational training.

More specifically, the program envisions working closely with local stakeholders in supporting local TVET and employment stakeholders as an effective and sustainable way to enhance employability. The program targets mainly disadvantaged categories of the population by improving their skills and competencies in line with labour market requirements and increase their income generating capacity by supporting them in the employment or self-employment.

Rwanda, Burundi and South Kivu in the Democratic Republic of Congo are historically, geopolitically and commercially interdependent, they have regular formal and informal exchanges of workforce and they face similar challenges regarding youth unemployment due to lack of access to market oriented skills training, weak private sector, little industrial development, non-adapted Technical and Vocational Education and Training systems, as well as lack of clear qualification frameworks. To overcome these challenges, these countries have prioritized promotion of vocational training in their development strategies. PROMOST started in Rwanda, where it reached the expected outcomes (external evaluation in 2015); a second phase built on these results in Rwanda and expanded the program to Burundi and South Kivu with greater focus on the private sector.

The program has the following key objectives:

                      I.        Increase accessibility for rural population and expand vocational skills offer;

                     II.        Improve quality and quantity of available training provision with special emphasis on disadvantaged youth groups and women;

                   III.        Develop the institutional capacity of the relevant institutional system and its key actors;

                   IV.        Contribute to establish a certification system valid in the whole regional labour market: East African Community (EAC) and the Economic Community of the Great Lakes Countries (CEPGL);

                    V.        Support the involvement of the private sector in the TVET-sector in offering internships and apprenticeship for TVET-students and placements for graduates and providing continuous trainings for its employees.

3. Subject of the evaluation/review: TVET Program Phase  II                                 

The implementation of the first two phases of PROMOST was mandated to Swisscontact after an international tender process in 2012. At the end of phase 1 in 2015, an external evaluation was done, whose findings were largely positive. The main conclusions recommended, among others, the pursuit for enhancing quality of training delivery. The present evaluation is meant to assess the outcomes achieved, the challenges met and provide forward looking recommendations for the design of the next third phase.

3.1. Overall program goal

To contribute to increased employment and income generation for the rural population in the great lakes by improving their access to quality and labour market oriented vocational training

3.2. Expected Outcomes:

                      I.        Access: At least 5700 persons are trained by end of 2018 of which 40% are women and girls and 2400 received an entrepreneurial training;

                     II.        Quality: At least 15 private training structures are equipped and supported (Burundi and S-Kivu) by the end of 2018, one more public VTC centre is built equipped, and fully operational in Rwanda;

                   III.        Capacity: 50% of the agricultural and artisan masters have expanded their technical and     entrepreneurial competencies in Burundi and Kivu by end of 2018;

                   IV.        Certification: Certification standards for selected priority trades are harmonized and validated by the authorities;

                    V.        At least 30 new professional associations are established in both countries by the end of 2018;

                   VI.        A regional TVET thematic platform is established and synergies among stakeholders including SDC regional offices are developed.

3.2. Target groups:

The main target group is the rural population of the great lakes region with a special focus on disadvantaged groups and women, excluded from formal TVET. Other direct beneficiaries are trainers, public policy implementers, teachers and the administrative staff in targeted TVET institutions.

4. Approach and scope

4.1. Approach

In view of the fact that TVET Program implementation began in December 2012, the approach will be a Forward looking evaluation to determine the extent to which anticipated results were realized, focus on providing information about the impact of the program and look into how best we can shape the next phase, to achieve sustainability and a greater impact.

4.2. Scope

The breadth and depth of the evaluation will be informed by the questions that the evaluation seeks to answer (see “key focus areas” below section 5). The evaluation will assess overall program implementation, as well as the implementation of its parts, in light of the region’s current political, economic and social context. The time frame is limited the present phase, from 01.02.2016 to 30.01.2019.

5.  Key Focus Areas

Before the start of the evaluation, the team leader will be required to draft an evaluation design matrix with sub-questions that will guide the evaluation. The questions set out below, were grouped according to the main objectives of the evaluation, and are intended to provide overall guidance to the drafting of the evaluation design matrix and the sub-questions referred to earlier. The evaluation mission will assess the SDC’s contribution to vocational and skills development in Rwanda, Burundi and Bukavu, according to the following standard evaluation criteria:

5.1. Effectiveness

                        I.        To what extent has the TVET Program contributed to a demand-oriented TVET system in the region?

                       II.        Are the target’s group aspirations, expectations and specific needs met?

                     III.        Which capacities of stakeholders were changed, to what extent and how?

                     IV.        To what extent has the project contributed to employment and income generation?

                      V.        What are the major factors influencing the achievement or non-achievement of the outcomes?

5.2. Relevance

                      I.        Is the project relevant in relation to the needs and priorities of the intended beneficiaries?

                     II.        To what extent the program and the curricula reflect and/or respond to the requirements of the economies of the region?

                   III.        Are the project results consistent with the targeted rural area needs and priorities of the respective countries (Burundi, Rwanda and South Kivu-Bukavu)?

                   IV.        To what extent are the objectives of the project still valid?

                    V.        Are the activities and outputs of the project consistent with the overall goal and the attainment of its outcomes?

                   VI.        To what extent has the quality offer of training been improved?

5.3. Efficiency

                      I.        Were the activities of interventions cost-efficient, i.e. how economically have resources/inputs (funds, expertise, time, etc.) been converted to results?

                     II.        Have the objectives been achieved on time?

                   III.        Has the project been implemented in the most efficient way compared with alternatives?

                   IV.        Is the project “set-up” adequate? Inclusive? Transparent? Flexible?

5.4. Sustainability

                      I.        How sustainable is the program?

                     II.        To what extent are the benefits of the project likely to continue after SDC funding ceases?

                   III.        To what extend is local ownership established?

                   IV.        What is the uptake by the Government of Rwanda, specifically by the Ministry of Education (MINEDUC), based on project results to upscale or replicate?

                    V.        How can we influence the vocational training system in a province like South Kivu and in a country in crisis like Burundi?

                   VI.        Are the strategies chosen relevant in these contexts? What are the major factors that could be identified at this early stage that influences the achievement or non-achievement of the project’s sustainability in the region?

6. Methodology

The evaluation team is expected to suggest a proper methodology. Overall, it is suggested to use participatory approach. The review team (consisting of the international TVET expert as evaluation Team Leader, a local VET expert with M&E expertise) will closely collaborate with the main stakeholders in order to ensure that the findings of the evaluation are credible, sustainable and likely to be used. Under the guidance of the SDC Office in Kigali, the review team will work together with the program implementation unit (Swisscontact), government and private sector partners. Thorough preparation ahead of the regional mission is a key requirement. Before the mission, the team leader is expected to deliver an evaluation approach paper with a draft work plan for the evaluation (including the evaluation design matrix) in collaboration with evaluation team members. The work plan will be based on a desk review of reference documents (“Reference documents”) and communication sessions with SDC Kigali. During the mission, the review team will interact and hold interviews with partners, stakeholders and beneficiaries, visit implementation sites, and facilitate participatory workshops and discussions among stakeholders both inside and outside of the program to determine their views about the context and implementation of the project.

At the end of the mission, the review team will hold a debriefing with SDC management in Rwanda. Findings and recommendations will be captured in the evaluation report, which is to be submitted and shared after the evaluation mission.

7. Review team

The mission core team will be conducted by one international and 3 local experts (1 from Rwanda, 1 from Burundi and 1 from Bukavu) with context specific experience.

7.1. Team leader

The team leader will be an international expert and will assume overall responsibility for the expected results of the review mission. Requirements: s/he will be a senior professional with proven experience (including overseas) and expertise in all aspects of program-cycle management: Project planning/design, implementation, and in particular a proven record of program evaluations/reviews. In addition, he/she is expected to have a thorough understanding of development processes, including both technical and capacity building. Understanding and experience with TVET systems is a must, as are sound interpersonal and report-writing skills. Experience in labour related policies is desirable. Previous experience as an evaluation team leader is a key requirement.

7.2. Local TVET experts

Professional with proven experience and expertise in all aspects of TVET systems, experience in program evaluations/reviews is an advantage. In addition, the experts are expected to have a sound theoretical, monitoring and practical understanding of the Rwandan, Burundi and/or DRC’s TVET systems. The local experts are expected to support the team leader by: (1) Setting up the mission schedule and logistics; (2) Contextualizing findings, conclusions and recommendations; (3) Providing sector insights; and (4) Providing written inputs as requested by the team leader.

8. Deliverables/Outputs

The Mission will complete and submit a draft final report in both hard and soft copy at the end of the mission. The mission’s findings and recommendations will be thoroughly discussed with SDC, program partners, and the program management. The mission leader will finalize the report in the light of comments/suggestions of stakeholders.

The key deliverables of the evaluation are:

              I.        Draft Report: Submission of a draft report format containing Table of Contents for the final report for approval by SDC.

             II.        An Aide Memoire (including key findings and recommendations) and its presentation: The Mission will present the aide memoire to the project stakeholders (SDC, and SC) in a consultative workshop.

           III.        The evaluation and Final Reports: The report should be logically structured, contain evidence-based findings, conclusions, lessons and recommendations, and should be free of information that is not relevant to the overall analysis. The report should respond in detail to the key focus areas described above. It should include a set of specific recommendations formulated for the project, and identify the necessary actions required to be undertaken, who should undertake those and possible time-lines (if any).  Project stakeholders will provide comments on the Draft Report, and the mission will finalize the report in view of these comments.

           IV.        For presenting and discussing the draft final report interactively, the consultants will facilitate a one-day concluding workshop for the project stakeholders and recommendations should come with strong justifications.

The report should be developed in view of the need to:

              I.        Validate the current project outcomes and report on project’s progress in relation to the anticipated outcomes, both within the existing constraints and in relation to its consequences for the final year of implementation.

             II.        Identify the gaps to be addressed, lessons learned to be applied, and any adjustments to be made in relation to outputs, outcomes and strategies, and actions needed to achieve these adjustments.

           III.        Provide recommendations on new activities and upscaling/improvements within the scope of existing activities, approaches and strategies to be undertaken by the project to achieve the expected outputs and outcomes.

           IV.        Provide recommendations for improved collaboration and cooperation with the relevant government agencies, implementing partners and donor organisations in order to increase the effectiveness of the project components in support of the overall government’s TVET reform agenda.

            V.        Provide indications for possible implementation options of a TVET Project Phase III from 2019-2022.

           VI.        The team Leader is responsible for preparing the final report, with written inputs from team members. He/she is expected to propose a draft outline/structure for the evaluation report ahead of the mission for approval by SDC. The outline/structure will be again validated at the end of the mission during the debriefing.

          VII.        The final report should be sent to the Swiss Regional Director of Cooperation in the Great Lakes, Kigali office, Rwanda. It should be a detailed report of no more than 40 pages (excluding annexes), written in English.

9. Timelines

The following plan is only indicative:

The evaluation will begin in May 2018. Timeframe to be discussed with consultant(s), but the work will be undertaken over an approximately 1 month timeline.

10. Budget

The budget(s) will be separately annexed to the contract. Contracts will be concluded with the team leader based on 31.0 working days, with the local TVET experts. International travel will be in economy class. Hotel costs (effective) and per diem for the region will be based on the guidelines of the Swiss Federal Department of Foreign Affairs. The team leader will present a breakdown of these days as well as their division of labour.

11. Reporting

The review team will report to the Director of Cooperation of SDC GL and her Deputy, in Kigali for the entire duration of the assignment. During the in-region mission, team members will work under the guidance of the team leader. Operational support will be provided by the country Cooperation offices and the National Program Officers (NPO) in the three countries.

12. Reference documents

      I.        Swiss Cooperation Strategy 2017-2020 (SDC GL)

     II.        TVET Policy(Rwanda, Burundi and DRC ) and strategy

   III.        TVET Program Document phase II (SDC)

   IV.        TVET Credit Proposal document phase II (SDC)

    V.        Annual Plan 2016-18

   VI.        Annual Reports:2016-2018

  VII.        TVET Laws

VIII.        SDC Guidelines for Basic Education and Vocational Skills development

   IX.        Other documents and regulations as deemed necessary

13. Schedule of Payment

The evaluator will receive an initial payment of 50% of the total contract amount (enough to cover travel and accommodation costs) which is due upon signature of the contract. Final payment will be made upon satisfactory completion of work. The fee is inclusive of all taxes, and expenses such as travel, accommodation and incidental expenses.

In case, the evaluator cannot provide the products in accordance with the ToRs, the timeframe agreed, or if the products are not of the required standard and quality, the payment to the evaluator shall be withheld, until the products are modified to meet SDC's standards. In case the evaluator fails to submit a satisfactory final product to SDC, the product prepared by the evaluator may not constitute the evaluation report and the contract shall be terminated.

14.  Evaluation Criteria

15.  Application Procedure

Technical and financial proposals are requested from interested consultants to be sent to; caroline.tissot@eda.admin.ch or claude.rwagitare@eda.admin.ch by 16:30 pm on 30.03.2018. The technical proposal should not exceed five pages and should outline the service provider’s:

                       I.        Understanding of the assignment;

                      II.        Approach to the assignment;

                     III.        Experience with similar assignments;

                    IV.        Draft evaluation work plan; and

                     V.        Draft report outline.

The financial proposal should be no more than one page and should clearly outline the daily rates in Swiss Francs (CHF). It is not necessary to include travel costs, hotel costs and per diems in Rwanda, Burundi and Bukavu for the financial proposal at this stage.

Interested consultant(s) is/or are required to include 3 local TVET experts with context specific experience (1 for Rwanda, 1 for Burundi and 1 for DRC) in their technical or financial proposal. The local TVET experts are contracted separately and directly by the team leader.

The technical and financial proposal should be accompanied by annexes including a CV and references from similar assignments, mentioning the names of two to three referees who can provide evidence of competence in leading similar evaluation assignments.

16. Contracting

The contract will be awarded by the Swiss Cooperation office after a joint conclusion and analysis of technical and financial proposals received in response to these terms of reference. Once winning the tender, the consultant (s) should submit their inception report showing how they interpret the terms of reference as well as Technical and financial proposals to Swiss Cooperation office in Kigali. Technical and financial proposals can be submitted to Swiss Cooperation either by: e-Mail to: caroline.tissot@eda.admin.ch, and/or claude.rwagitare@eda.admin.ch or by hand to Swiss Cooperation offices, in Kigali, before 16:30 pm local time on 30.03.2018. Interviews will take place on the 10.04.2018.