South Sudan: Mid-Term Evaluation Food Security Enhancing through Sustainable Agriculture Project

Project location: Gogrial East and Gogrial West Counties

Warrap State

South Sudan

Project Contract Period: August 1^st 2014 to July 31^st 2017

February 2016

The project is funded Implemented and Managed Co-implemented by Serve Education

by the European Union by Veterinaries sans Frontiers Germany Development Association

Implementation supported by INFRAID (Associate)

Glossary

CAD County Agriculture Department

CAHW Community Animal Health Worker

CLD County Livestock Department

FFS Farmers Field School

FGD Focus Group Discussion

FS Food Security

GOSS Government of the Republic of South Sudan

HH Household

IDP Internally Displaced Person

INFRAID Indigenous Forest resources Awareness and Improvement of Diet

INGO International Non-Governmental Organization

KII Key Informant Interview

LRRD Linking Relief, Rehabilitation and Development

NFI Non Food Item

QA Quality Assurance

ROSS Republic of South Sudan

RRC Relief and Rehabilitation Commission

SEDA Serve Education and Development Authority (SEDA)

SMAF State Ministry of Agriculture and Forestry

SMARF State Ministry of Animal Resources and Fisheries

TOR Terms of Reference

VSF G Veterinaries San Frontiers - Germany

Table of Contents

1.Project Summary. 6

2.Background and context. 7

2.1.Overall description. 7

2.2.Project objectives and results. 8

3.Evaluation purpose and target audience. 9

4.Evaluation objectives and scope. 10

5.Evaluation tasks and questions. 11

6.Approach and methodology. 12

7.Timing and deliverables. 13

7.1.Inception report. 13

7.2.Draft mid-term evaluation report. 13

7.3.Final mid-term evaluation report. 13

7.4.Roles and responsibilities. 14

7.5.Tentative itinerary. 15

8.Evaluation team composition and required competencies. 16

9.Management arrangements. 16

10.Budget and payment. 17

11.Proposal submission. 18

1.Project Summary

Contract Beneficiary

Vétérinaires Sans Frontieres Germany

European Union Programme

Food Security Thematic Programme (FSTP)

Project Title

Food Security Enhancing through Sustainable Agriculture Production (FESAP)

Location

Gogrial East and West Counties, Warrap State, South Sudan

Project Duration

8^th August 2014 to 7^th July 2017

Project Phase

Second Phase

Assessment Type

Mid Term Evaluation

Evaluation Purpose

To document and inform the stakeholders (donors, partners and beneficiaries) on the program’s achievements so far against set objectives and targets with due reference to relevance, effectiveness, and sustainability; the potential impact in relation to relevant VSF Germany policies and EU standards, the challenges and the lessons learned for project improvement in the next period.

Evaluation Objectives

To assess the overall performance of the project against the baseline data established at the start of the project.

To assess progress in the achievements of outcomes, outputs and activities

To assess the scope, quality and relevance of the project outputs produced so far in relation to the project theory and design

To provide actionable, context appropriate and innovative recommendations for project improvement in the next implementation period

Methodology

Review of available secondary data

Focus Group Discussion and key informant interviews

Household survey

Observation

Evaluation Start and End dates

1^st to 15^th March 2016

Anticipated Evaluation report release dates

17^th March 2016

2.Background and context

2.1.Overall description

a)Background

Vétérinaires Sans Frontières Germany (VSF-G) is an international Non Governmental Organization, providing humanitarian aid and development assistance to pastoralists and vulnerable communities in areas where livestock is of importance. VSF-G support is in animal health; livestock related agriculture, marketing, food safety, drought responses and mitigation, capacity development of communities and governmental institutions, peace and conflict resolution with the ultimate aim of food security and strengthened livelihoods of pastoralist communities.

In South Sudan VSF-G has been implementing a European Union (EU) funded project - Food Security through Enhancing Sustainable Agricultural Production (FESAP) – in partnership with its local nongovernmental organization partners, Serve Education and Development Authority (SEDA) as a co-implementing partner and Indigenous Forest Resources Awareness in the Improvement of Diet (INFRAID) as an associate partner. Implementation of the four year project started in August 2014 and is scheduled to end in July 2017. The project is being implemented in Gogrial West and Gogrial East Counties of Warrap State of the Republic of South Sudan. The total population of the project catchment area is 717,331 (population census 2008) and is predominantly agro-pastoral.

FESAP is funded under the European Union’s Food Security Thematic Programme (FSTP) which supports activities aimed at improving food security for the world's poorest and most vulnerable populations. It addresses structural issues leading to hunger and under-nutrition in a bid to resolve these in the medium or long term, with an emphasis on developing sustainable solutions. FESAP aims to contribute to this global objective by building capacities of local farmers and cattle keepers (returnees, IDPs, and host communities) and other actors using market-oriented approaches to improve food production and household incomes.

b)The 4 key result areas of the Project are:

Result 1: Increased agriculture production and productivity through strengthened extension services and innovative farming.

Result 2: Animal traction is adopted as a viable, cost-effective and sustainable technology and contributes to annual increase in cultivated farmed land, improved tillage and increased productivity per acreage.

Result 3: Increase in household income as a result of access to microfinance, local markets and linkages to value chain actors.

Result 4: Increased resilience to shocks in climatic/environmental changes and manmade disasters such as conflict.

A baseline (formative) evaluation was carried out and completed in February 2015 and EU project monitoring missions carried out in October 2014 and November 2015. The first year interim report covering the period 8h August 2014 to July 7^th 2015 has been submitted to the EU.

The midpoint of three year FESAP project will be March 2016, for which matter VSF G is seeking to conduct a mid-term evaluation of the project progress made to date against the baseline. The review will be participatory with valid inputs from all the relevant stakeholders such as beneficiaries, local and state agriculture and livestock officials**,** VSFG Staff, SEDA, INFRAID and other community stakeholders.

2.2.Project objectives and results

Overall Objective: To contribute to increased food security, reduced vulnerability and enhanced livelihoods of rural communities by supporting household subsistence farmers in Warrap State of South Sudan.

Specific Objective1: To increase sustainable food production and productivity through effective extension services and promotion of animal draught technologies.

Specific Objective 2: To increase income of farmers through promotion of market oriented farming and facilitation of access to credits, markets and linkages between value chain actors.

Specific Objective 3: To improve and strengthen the capacity and capabilities of target groups to respond to vulnerability/shocks caused by climatic, environmental and other factors including man-made conflicts.

Results 1: Increased agriculture production and productivity through strengthened extension services and innovative farming

Outcome

% of population who improved their food security status.

Activities

· % of trained progressive farmers and extension workers (disaggregated by gender/community/ private/ government) sharing their knowledge with other farmers in the community

· % of farmers purchasing regularly inputs (vet and agro products) and services from the agro-vet shops and other private dealers.

· # of HHs targeted practicing improved farming methods ( e.g. planting in rows)

· % increase of quality seed availability at community level due to seed multiplication systems.

· # of target private extension service providers continuing to function during and after the project

Result 2. : Animal traction is adopted as a viable, cost-effective and sustainable technology and contributes to annual increase in cultivated farmed land, improved tillage and increased productivity per acreage;

Outcome

At least 30% of targeted and supported famers continue using animal traction (cultivation) without external support

Activities

· % increase in number of animals used for traction by the end of the project in target location

· % increase in cropped land in the farm (production) by the end of the project

· % increase in understanding of Ox-plough utilization and management by farmers

Result 3.Increase in household income as a result of access to microfinance, local markets and linkages to value chain actors.

Outcome

· % of Increased household productive assets for target beneficiaries

Activities

· % Increase in levels of savings/borrowings through microfinance by the end of the project.

· % of subsistence farmers increases by 20% the profit from sale of agricultural products by the end of the project.

· % of the self-saving groups formed sustainably, running and productive.

Result 4 : Increased resilience to shocks in climatic/ environmental changes and manmade disasters such as conflict

Outcome

% increase in food availability of the target beneficiaries

Activities

· % reduction in emergency selling of productive assets in dry season by the end of the project

· # of seed banks supported and operational in target communities

· % of subsistence farmers (disaggregated data by gender) using improved harvest storage system

· % of the supported households have better nutritional balance due to improved diet

3.Evaluation purpose and target audience

3.1.Evaluation type

This is an interim evaluation to assess the progress made by the project towards achieving planned activities and results as set out in the project design.

3.2.Purpose

The purpose of the mid-term evaluation is to carry out an overall appraisal of the efficiency and progress of project towards meeting its objective so as to provide decision-making information for use to improve the quality and relevance of the project and its implementation; and to identify necessary reorientations and follow-up actions in the remaining period to ensure the achievement of the original objectives.

3.3.Target audience

a.The European Union

The mid-term evaluation shall provide the European Union with information on how project is performing towards finally contributing to the objectives of the FSTP programme and what follow-up actions may be necessary to ensure the desired outcomes.

b.VSF Germany and partners

The mid-term evaluation shall provide VSF G and its partners opportunity to critically assess administrative and technical issues and mechanisms and use of the resulting recommendations to improve the potential of FESAP to achieve expected outcomes and objectives within the project timeframe.

c.Beneficiaries and stakeholders

The mid-term evaluation process will accord beneficiaries and stakeholders opportunity to participate in the review process, build the capacity for effective participation in reviews, and enhance ownership and accountability of the implementing partners. The exercise will enable improvement of the relevance of the project to the needs of the beneficiary and priorities of the stakeholders.

4.Evaluation objectives and scope

The overall purpose of the evaluation is to assess the progress made towards achieving expected outputs and outcomes and identify and document lessons-learned and to make recommendations to improve the project.

4.1.Evaluation objectives

a. To appraise and establish the relevance of the project

The evaluation will appraise the appropriateness of project objectives to the problems that it was supposed to address, and to the physical and policy environment within which it operates. This will include an assessment of the quality of project preparation and design – i.e. the logic and completeness of the project planning process, the internal logic and coherence of the project design.

b. To determine the level of efficiency of the project implementation

Project efficiency will be evaluated by gauging whether the progress made towards achieving the expected results is happening at a reasonable cost – how efficiently Means and Activities are being converted into Outputs.

c. To determine the effectiveness of the project implementation

The mid-term evaluation will determine the contribution made by project Outputs towards achieving the Project Objectives, and how Assumptions have affected project achievements.

The evaluation shall also determine the effectiveness of the employed approaches in addressing the primary challenges affecting the community. The linkage and resultant synergy between these respective components shall be appraised.

d. To determine the project quality

The evaluation shall appraise FESAP against desirable project quality factors that include: Participation and ownership by beneficiaries; Policy conformity; Appropriate technology; Socio-cultural implications; Gender equality; Environmental protection; Local institutional development; and Management capacities; and Financial and economic viability. Any deviation from initial plans shall be evaluated and accounted for. Achievements made in respect of the exit strategy will be gauged.

4.2.Scope of the evaluation

The mid-term review period will focus on the past 1 ½ years of project implementation from August 2014 to Feb 2016 in the targeted project areas in Gogrial West and Gogrial East Counties. The evaluation will cover in detail programming quality as well as protection and equality topics.

5.Evaluation tasks and questions

The evaluation tasks relate directly to the evaluation objectives and should be comprehensively addressed with reference to one another.

5.1.Determine the relevance of the project

a. What is the prevailing project context and its significance to project implementation?

b. Who are the key actors (stakeholders) in the project, what are their roles and responsibilities and how have they influenced project implementation?

c. What is the relevance of the project objectives to the beneficiary, community, governance and policy priorities at project and how has this changed during the life span of the project?

d. What are the strengths and weaknesses in project design and how have these impacted on project implementation and monitoring?

5.2.Determine the efficiency of the project

a. What is the level of cost-effectiveness in the accomplishment of the outputs completed so far or in the process of completion?

b. What is the quality of the outputs?

c. What are the strengths and successes in project implementation (management, staff, coordination and reporting)?

d. What are the weakness and challenges in project implementation (management, staff capacity, coordination and reporting)?

5.3.Determine the effectiveness of the project

a. To what extent is the project meeting its objectives?

b. To what extent has each of the approaches employed by the project contributed towards achievement of project objectives?

c. What are the identified and potential barriers to successful delivery of the project?

d. How have the assumptions made at project design evolved and how have they impacted on the project implementation and progress towards achievement of the objectives?

e. What is the degree of project quality with reference to the evaluation’s desired project quality factors?

5.4.Determine progress towards impact and sustainability of results

a. What is the degree of soundness and feasibility of the projects exit strategy and to what extent has it been implemented?

b. What is the level of overall viability of the expected outcomes and signs of sustainability of the same?

6.Approach and methodology

The assessment methodology proposed by the consultant(s) should include a mix of quantitative and qualitative data collection methods. The desired methodology is participatory and all the key stakeholders in the program including target beneficiaries, County Agriculture and County Livestock Department staff, will be considered to be part of the mid-term evaluation process. The evaluation will be based on the findings and factual statements identified from review of relevant documents including the project document, Annual Project Reports (APR), Project Implementation Reports (PIR), in addition to the technical reports produced by the project.

The anticipated benefit of a Participatory and stakeholder-centred approach is the empowering that the process will impart to those service providers and the beneficiaries/ stakeholders who will participate in the exercise. Focus group discussions, key informant interviews, site visits etc are proposed amongst the methods to be used for the review. The consultant will provide leadership and bear responsibility for the process, the findings, the comments and the content of the final document. VSFG will be responsible for organizing the locations for conducting the Key Informant interviews, household questionnaire and Focus Group discussions in the respective payams of operation whilst the consultant will review and finalize the tools in conjunction with VSFG Programming team. Participation of stakeholders in the evaluation should be maintained at all the times, reflecting opinions, expectations and vision about the contribution of the project towards the achievement of its objectives.

The general rule related to acceptable margins of error (5%) in social research will apply to the study. Sampling criteria should be agreed upon with VSFG National office, a 30 x 30 cluster sampling is recommended, however, the Consultant may have the discretion to employ any other sampling methodology upon giving appropriate justification which will then be reviewed by VSF Germany programme team before being adopted.

7.Timing and deliverables

7.1.Inception report

The inception report shall provide a detailed description of the methodology to answer the evaluation questions as well as the proposed source of information and data collection procedure. The inception report shall outline the contents of all the deliverables.

7.2.Draft mid-term evaluation report

The consultancy team will develop draft evaluation report for review by the FESAP programme personnel and partners. The report shall be validated by stakeholders in a workshop setting and feedback incorporated into the report. The draft report will adopt the format of the final report as presented below under the final report. Generally report will include the Executive summary, Intervention description, Evaluation purpose, Evaluation methodology, Findings, Conclusions (answers to the Evaluation Questions), Recommendations and Annexes (list of people interviewed, key documents consulted, data collection instruments, summary of the TOR). Further details will be found in the annexes.

7.3.Final mid-term evaluation report

The consultancy team shall endeavor to develop the final report and present the output in an electronic format to VSF Germany for final approval and adoption. The final report (Word, Excel files to be put in PDF as well) will be submitted according to the evaluation timeline. The report will have the following structure:

1. Cover page (1 page)

2. Table of Contents (1 page)

3. Acknowledgements (1 page)

4. Glossary (1 page)

5. Introduction (1 page)

6. Project being evaluated (1 page)

7. Executive summary (2 Pages)

8. Evaluation introduction/Background (max 2 pages)

9. Methodology (max 2 pages)

10. Findings (max 10 pages)

11. Conclusion and recommendations (max 2 pages)

12. Lessons learnt from the evaluation process (max 1 pages)

13. Appendices (to include copies of all tools, list of enumerators, survey timeline including all KII and FGD participants and discussion transcripts, (as many pages as necessary- please reference the annexes in the report, but include them in a zip file as separate documents.

The final report will be accompanied by the following deliverables:

· A 2 page evaluation fact sheet and soft copy of dataset. This is to include relevant findings from the evaluation, key points and recommendations.

· An Indicator Summary Sheet, giving status of all indicators measured in the HH questionnaire compared against baseline values

· Learning dossier - Lessons learnt by the entire evaluation team shall be documented and shared with the project team and Programs Office Juba so that they may be taken into consideration for future studies. The documentation of these lessons will be vital for reflection, growth and continued improvement. The lessons will be drawn from the process and the actual house hold surveys, key informant interviews, focus group discussions, study observations and secondary data reviews.

7.4.Roles and responsibilities

Each member of the evaluation team shall have designated roles and responsibilities for the delivery of the evaluation.

Scope of Work and responsible parties

a)Evaluation tools are properly developed and in line with VSF G and EU evaluation standards

· Review of the available relevant project documents i.e. project proposal, log frames.

· Coordinate HH Interviews for selected respondents during the evaluation study.

· Training of enumerators.

· Conduct interviews with selected partners, staff and other

· Visit selected project sites.

Responsible: Consultant, Program Manager

b)Coordination of the household interviews and well us providing guidance to the focus groups discussions

· Household interview and FGD participants are properly selected

· Guide the FGD and interviewers on proper data entry and documentation/filing

· Prepare the summary reports of each FGD as per agreed upon format.

· Ensure completeness of the survey instruments questionnaires

Responsible: Consultant, Project Manager

c)Apply effectively the quantitative and qualitative tools

· Obtain consent from the household or FGD members to conduct the evaluation.

· Record household data on questionnaire as per training instructions.

· Record all key notes related to issues arising while interviewing.

· Review and submit the survey data to survey team leader in accordance with the schedule

Responsible: Consultant

d)Provide overall coordination, logistics, and technical oversight in the Evaluation process.

· Brief stakeholders about the purpose of the evaluation.

· Brief stakeholders about the purpose of the evaluation.

· Provide technical oversight and monitoring of the Survey and ensure timely completion and compliance with international evaluation standards.

· Avail all project related secondary data.

· Avail all the required logistics including vehicles for the Evaluation.

· Assist in organizing meetings with stakeholders.

· Avail all the required logistics including vehicles for the Evaluation.

· Recruit and pay the evaluation enumerators.

· Supervise adequate utilization of the resources allocated for the study.

· Budget preparation and management during the evaluation period/ensuring that all expenses are properly documented.

· Budget preparation and management during the evaluation period/ensuring that all expenses are properly documented.

· Oversee accommodation and meals for data collection teams.

Responsible: Project Manager and Consultant

7.5.Tentative itinerary

Event

Responsible Persons

Timelines

Preparation phase

Desk phase – secondary data review, tools development

Review – secondary data, information

Consultant

1

Data Collection tools Development

Share Data Collection tools with VSF Germany programs

Consultant

Consultant

2

Field phase – data collection

Enumerators selection and Training

Consultant

2

Pretesting, refining and printing of evaluation tools

Consultant & project staff

1

Data collection and interviews

Consultant and Evaluation team

6

Synthesis phase

Data Analysis and validation

Consultant

2

Draft Report of Evaluation

Consultant

1

Final Report with Feed back

Consultant

1

Total Number of days excluding preparation and planning days

16 Days

8.Evaluation team composition and required competencies

The evaluation team shall compose of members with a comprehensive mix of competencies in agro-pastoral production and market systems and social research methodologies. These will be complemented with at least five years’ experience in related programming and programme research. Extensive experience in the specific fields in the horn of Africa and South Sudan in particular will be required. Excellent communication skills and well as demonstrated writing and presentation skills are requisite.

9.Management arrangements

The consultant should be informed of some issues, situation and conditions as they are or may arise during the exercise.

a. *Travel:* All international flights land in Juba, it is not possible to fly to Warrap on the same day. Warrap flights are only during week days. The international airport of Juba is closed on Saturdays and Sundays. The consultant should take in to consideration this challenge that should not lead to cancellation of the exercise. VSFG will cover the cost of all internal flights and transport.

b. Accommodation: Consultants will be housed in hotels in Juba and Warrap whilst in Luonyaker they will be housed at the organizations compound lodgings. However, electricity for powering laptops is not guaranteed at all times. Internet access will be available at VSF Germany offices, but may not be available in the hotels.

c. Data entry may not be possible in the field. VSF G will not supply data entry clerks or equipment for data entry. Consultants are responsible for all data entry and management. All hard copies of tools will need to be transported by consultants to the place where data entry will be done. All data sets must be provided to VSF G in soft copy at the time of submission. They are the property of VSFG and the communities from which the data will be collected and may be used for future analysis.

Data will in most instances be collected from non-English speakers. However, tools will not be translated into the local language. A way around this matter will be developed in discussion with project staff, VSF G program staff and the consultant.

d. Poor documentation- in terms of secondary data: Updated information has been noted to be a challenge during literature review exercises especially from government departments and other community based organization. Therefore, there may be a level of inconsistency in the information provided and/or collected.

In addition, with the frequent relapse of inter-ethnic conflict in the region, the security situation might be a hindrance to the evaluation process as this could limit meetings and travelling within the region. This may also delay the timely delivery of final product.

e.Operation arrangement

· Accommodation and transport will be provided by VSF Germany;

· Translators, drivers, facilitators, office space, printing of questionnaires etc will be provided by VSF Germany;

· The contact person in South Sudan will be the Country Programme Manager;

· The focal person in the field will be the Project Manager;

· Security advisory issues will be provided by VSF Germany;

· VSF Germany will take care of internal travels but in case of international flights, the consultant will organize and VSF Germany will pay reasonable prices incurred only.

f.Reference materials

Relevant documents will be availed for the consultant to support during the secondary information desk reviews. The consultant will be encouraged to identify any other sources for appropriate additional information that may be required to supplement what is provided by the project.

The Project Team will share the following documents with the Consultants for reference.

1. Full project proposal

2. Project agreement contract

3. Baseline report

4. Project log frame

5. Project activity reports

6. Project first year report

7. Sudan Population and Housing Census report 2008

8. National Baseline Household Survey 2009

9. FAO/WFP food security assessment/survey reports

10. South Sudan Development Plan 2011-17

11. Humanitarian Response Plan 2015, 2016

12. South Sudan Infrastructure Action Plan, Chapter 6

13. Investing in Agriculture for Food Security & Economic Transformation – UNDP

10.Budget and payment

The evaluation budget will take care of the expenses such as the consultant’s charges, international travel, in country travel, accommodation full board, and fuel for vehicles to be used during the survey period, enumerators’ fees and lunches, stationery and communication costs. VSF Germany will take care of the following

a) Flights (internal and international for the consultant)

b) Accommodation in South Sudan

c) Field transport (fuel and car rentals)

d) Enumerators fees

e) Stationeries

The following costs will not be covered by the organization and should be factored into the consultancy and related fee which the consultant will submit with the application.

· Costs for data handling, entry and processing

· Communication cost.

· Report writing and printing.

· Any medical expenses by the consultant during the assignment

1.Proposal submission

The proposals will be evaluated according to the following criteria;

a) Technical and financial proposal

b) Proposed personnel for the assignment

c) Profile of the applicant - corporate or individual

Proposals should be submitted electronically to the following Email addresses: juba@vsfg.org and admin_hr@vsfg.org indicating on the subject line thus “**MTE FESAP**”. The closing date for receiving proposals is 7^th March 2015. Only short-listed candidates will be contacted for interviews. The assessment of the proposals received will be on a continuous basis due to urgency of the assignment.