TERMS OF REFERENCE
MID-TERM EXTERNAL EVALUATION
TF-MADAD/2020/T04.255 – Prevention and Management of Non-Communicable Diseases through Primary Health Care. REAYAH
INDEX
1. JUSTIFICATION AND PROBLEM ANALYSIS. 3
5. EVALUATION STAKEHOLDERS MAPPING.. 11
6. WORKPLAN AND CHRONOGRAM.. 11
7. EVALUATION PRINCIPLES, AUTHORSHIP AND PUBLICATION.. 13
8. PROFILE OF THE EVALUATION TEAM.. 13
9. PARTICIPANTS AND SUBMISSION OF BIDS, BUDGET AND BIDS’ APPRAISAL CRITERIA. 14
10. APPENDIX I: Declaration of absence of conflict of interest 16
11. APPENDIX II. EVALUATION MATRIX MODEL. 18
Jordan, a middle-income economy, currently hosts 10,5 million people in its territory, having experienced a very fast population growth, due to a large extent to the arrival of refugees both from Syria as well as from other countries. Jordan currently hosts the second largest refugee population per capita in the world. As a result, an estimated 15% of the national population is made by Syrian, many of them refugees and out of the total population in the country 40% are refugees or immigrants from diverse nationalities.
Until the demographic and economic impact caused by the Syrian crisis, Jordan had developed a medium-high level health system, with fairly complete health legislation and good health outcomes: well-trained staff, a network of primary and hospital care services, and a series of subsystems of protection that were responding with a reasonable cost to the needs of the population. According to data from the United Nations (world population prospects) and data taken from the Annual Report of the MoH, life expectancy at birth had increased from 70.4 in 1995 to 74.4 in 2014. However, in recent years, it has stagnated, showing a slight decrease (73,5 accordingly to MoH reports).
Jordan has also achieved basic goals on maternal health and infant mortality included in the Sustainable Development Goals (SDGs). The proportion of births attended by health personnel exceeding 99%; the mortality rate of children under 5 years of 19 per 1,000 live births or the neonatal mortality of 11 cases per 1,000 live births. In the case of the proportion of women who make their own decisions and informed about sexual and reproductive health is 58%, still far from the universality proposed in the SDG.
During the recent period the economic restrictions imposed by the IMF have forced the reduction of public health expenditure, which in the General Budget Law for 2019 stands at 1,226.4 million JD. This would mean public health expenditure per inhabitant of 109 JD, compared to 141 JD in 2013.
Despite progress made by the Jordanian health system and the fact that Jordan is committed to move towards Universal Health Coverage (UHC), there were still in place structural challenges at a system level. The health insurance coverage is reaching overall around 55% of the population, while the percentage of women with any health insurance is 58,3 men are covered in a 51,7%. The lack of coverage translates into a lack of access to services for a significant part of the population and households. In the case of the Syrian population residing in Jordan the coverage is at a lower level (39,5 % women and 38,7% men).
A continued lower utilization of Primary Health Care-PHC service and preventive care has been reported as one of the reasons that threatens the financial health sector sustainability in the country and the PHC role as a gate keeper of the system. There are different reasons behind the decision of citizens to skip up PHC level and one of them could be related to the quality of the services delivered at public clinics. There is a lack of information and data regarding performance on the delivery of public health care services at this level as well as patient satisfaction.
Demographic evolution and epidemiological changes -increased incidence and prevalence of Non-communicable Diseases -NCD-, create new challenges for the health system. Determinants of poor health such as tobacco use, obesity, and other unhealthy behaviors are becoming increasingly prevalent in Jordan and are contributing to the increased incidence of NCD as well. Findings from Household Expenditure and Income Survey 2017-2018 (HEIS) indicate the percentage of people with chronic diseases is 12.6% among Jordanians while for non-Jordanians including Syrian refugees the figure is 11.1% at a national level. According to WHO, 78% of deaths are due to NCDs; in particular, cardiovascular diseases cause 37% of deaths; cancers 12% of deaths; Diabetes 6% of deaths.
The presence of Syrian refugees, of which more than 80% of those registered currently living outside camps among host community, have increased the demand and led to an increased pressure on the public health care system and, in particular, to an increase in rates of NCD and mental health problems, which hindered the response capacity of the health system.
In the context of the Syrian crises and their effects on the whole region, the EU Regional Trust Fund – EUTF in response to the Syrian crisis bundles an increasing share of EU assistance to the region into a single flexible instrument since its establishment in December 2014. The Fund primarily addresses the educational, economic and social needs of Syrian refugees, while supporting local communities and overburdened public administrations.
In line with the priorities established at the London Conference on Syria in 2016, the Brussels Conferences on the future of Syria and the region in 2017-2019, and the Global Compact on Refugees, the main objective of the Fund is to promote non-dependence of refugees, helping them thrive, not just survive, by closing the funding gap and the development-humanitarian nexus. At the same time, the Fund helps countries and communities hosting refugees to cope with the additional economic and social burden of the crisis.
Within the EUTF Operational Results Framework, the Fund contributes to better health for Syrian and host communities, through improved access to health services, strengthened local capacities and more resilient health infrastructure.
The Delegation of the European Union in Jordan approached the Spanish Agency for International Development Cooperation – AECID in 2019 to identify an intervention in Jordan aimed at strengthening the national capacities of the public health system to prevent and manage non-communicable diseases – NCDs at the national level of primary health care -PHC. During the period from June to October 2019, the identification and formulation of this intervention was carried out by the Spanish Cooperation and the Jordanian Ministry of Health. On March 30, 2020, the proposal was approved for an amount of €22 Million by the Fund’s Operating Board in Brussels and on the 14th of January 2021 the project started its implementation, for an initial period of 36 months, that has been extended in March 2023 for an additional 15 months.
According to the agreement, AECID is responsible for the implementation and justification of the entire intervention before the European Union, and different modalities and agreements with third parties have been established to comply with the operational implementation of the Action. The main national partner (subgrantee) is the Ministry of Health of Jordan, and there are complementary national partners (subgrantees) represented by Jordanian NGOs and private non-profit organizations (Royal Health Awareness Society-RHAS, Jordan Breast Cancer Program-JBCP, Institute for Family Health-IFH and Our Step Association) contributing to the Result 3 of REAYAH project. The intervention includes a component of technical support and exchange between the Jordanian and the Spanish National Health Systems, that contributes to achieve the outputs of Result 1 of REAYAH project, mainly.
As mentioned above, REAYAH project is part of the EU Regional Trust Fund – EUTF in response to the Syrian crisis and, as a such, contributes to its Operational Results Framework, specifically by contributing to better health for Syrian refugees and host communities, through improved access to health services.
This intervention aims to improve the efficiency of Ministry of Health (MoH) primary health care (PHC) for Syrian refugees and Jordanians when addressing non-communicable diseases (NCD) and mental health. For achieving this purpose, it is deemed important to strengthen MoH management and service providing capacities.
Consequently, this intervention supports the implementation of national health strategies on NCD; consolidate and strengthen the resilience of the public primary health care (human capital, premises and equipment) to treat outpatients by improving working conditions in health centers; better contribute to the prevention, early diagnosis and adequate treatment of NCD from the first level of attention; improving management tools at clinic level in a comprehensive and coherent way considering the current capacities at this level of care by MoH. Finally, also raising awareness among the population on healthy behaviors and risk factors.
INTERVENTION’S LOGIC
The overall objective is to contribute to the improvement of the health of Syrian refugees and Jordanians through improved prevention and access to strengthened PHC for NCD.
The specific objective of the action is to improve accessibility, cost efficiency and effectiveness of public primary healthcare (PHC) services in relation to NCD, notably in the three targeted governorates of Mafraq, Tafilah and Ajlun.
The expected Results and the correspondent indicators are as follow:
Result 1: The capacity of MoH PHC services in screening, early treatment, monitoring, referral and surveillance of NCD has expanded.
Result 2: Clinics in the governorates of Ajlun, Mafraq and Tafilah have improved their physical infrastructure, equipment and general management processes, offering conditions for improving the access of Syrian refugees and vulnerable Jordanians.
Result 3: Participation of targeted beneficiaries in prevention of NCD, promotion of physical and mental health and social support has been strengthened through raising awareness and engagement with patients, families, Health Community Committees, and community-based organizations (CBOs).
This intervention aims at improving the health care of Syrian refugees and host communities, strengthening the resilience of the public health system. Given the complexity of the Jordanian health system, this intervention does not intend to act comprehensively on all its aspects and parts (financing, coverage, portfolio of services, hospital sector, private sector, etc.). Instead, the intervention aims at intervening only in some of the aspects in need of support within MoH responsibilities, as they have been identified by different organizations, such as WHO, or the Panel of Experts of the European Union.
High impact essential NCD interventions can be delivered through a PHC approach to strengthen early detection and timely preventive treatment. Evidence shows that such interventions are excellent economic investments because, if provided early to patients, they can reduce the need for more expensive treatment like hospital admissions.
The activities included are taking advantage of what has already been done or is being implemented in the country. It is about strengthening staff capacities, process implementation methodology and evaluation, with the purpose of achieving a greater impact on the population as a whole, or on the selected populations. The approach is not to design new strategies but involves applying and expanding interventions that have proven cost-effective while creating internal change dynamics, and continuous improvement processes, mainly at a PHC level.
Considering the objective of the Intervention and the alternative chosen, the approach of the intervention is three-fold:
OBJECTIVES
The mid-term evaluation must address the impact of the intervention in progress according to the general logframe of the Action TF-MADAD/2020/T04.255. Assessing the implementation and performance of all national partners and AECID, based on the initial results and issuing suggestions on possible follow-up and corrective measures. This includes the revision on the coherence of the Log frame of the intervention, recommending possible adjustments, especially on the targets setup during the identification process.
The evaluation should provide AECID and national partners, especially the Ministry of Health, with a learning opportunity to improve upon existing practices by assessing the extent to which the outcomes of the project are being achieved, and determine the appropriateness, efficiency, effectiveness, coherence and long-term sustainability of the intervention.
The evaluation should help on identifying key elements that might support Reayah’s implementation, management, governance and decision-making in the remaining period of the intervention duration until beginning of 2025 according to the assessment on the current state of implementation.
The evaluation shall be based on objectivity, credibility, providing examples and programme recommendations, and identifying areas for improvement, all to enable AECID and, especially the Ministry of Health, to take corrective measures aimed at a better achievement of the project’s objectives and priorities. Recommendations for improvements of the current implementation shall be made considering the timeframe of the project, making possible for the concerned institutions to integrate them in the short-term.
The project is implemented mainly in 3 Governorates (Mafraq, Ajloun and Tafilah), having some components of capacity building at a national level. While the formal implementation of the overall intervention (contract EU-AECID) started in January 2021, sub-grants to national partners started implementation progressively during the second half of 2021 and are currently in different status of implementation and all of them contributing to the general log frame of the intervention.
Reayah’s stakeholders and sub-grants as on the date of the evaluation:
EVALUATION QUESTIONS AND CRITERIA
In consultation with AECID and after a preliminary documentary review, the evaluation team will revise the evaluation questions and analyze the explicit interest and feasibility of including them, so that they can be adjusted in a well-reasoned manner based on the information required, the evaluation deadlines and the resources earmarked for the evaluation. This adjustment will be included in the inception report, and should be validated by the management team.
The evaluation is expected to initially focus on the following questions.
Criteria 1 – Relevance, coherence, and validity
Is the project responding to the main needs of MoH health workers to improve their performance in terms of quality service delivery at primary health care in the field of NCD’s?
Is the project responding to the main needs of Jordanians and refuges to improve their access to primary health care in the field of NCD’s and reduce their vulnerability to NCDs?
Is the project aligned with the current priorities – reflected in national strategies and plans- of MoH in the field of primary health care and NCDs?
Have relevant cross-cutting markers, such as gender equality, human right to health or inclusion of people with disabilities, been adequately mainstreamed in the project design and implementation?
Criteria 2 – Effectiveness
How well the project design and implementation are contributing to better access, cost efficiency and effectiveness of public primary NCD healthcare services run by MoH in the prioritized regions? What progress has been made towards achieving the project results?
Has the management and governance structure put in place worked strategically with all key stakeholders and the donor to achieve project outcomes and objectives?
Is the monitoring and evaluation system results-based and facilitate a project adaptive management?
Has the programme considered external factors that might have been hampering (risks) or supporting (assumptions) results? Any unintended result (positive or negative) occurred?
Have all planned target groups been reached by the programme?
Criteria 3 – Efficiency
Is the resource input proportional to achieve results?
To what extent are the activities implemented as scheduled? What are the specific bottlenecks in implementation? How can they be mitigated?
How well does partner and stakeholder collaboration function? Is communication and coordination between partners satisfactory?
How well is communication and visibility integrated into the overall intervention and each specific result?
Criteria 5 – Impact and Sustainability
What real difference is the programme likely to make to participants and beneficiaries?
Are the results achieved likely to continue after the end of the programme and produce longer-term effects? What are the major factors influencing this?
What actions might be needed to boost longer-term effects?
Which project-supported tools and processes have been institutionalized, or have the potential to, by MoH?
To articulate in a systematic and logical matter the comprehensive judgement of the evaluation object, the technical proposal shall revolve around an evaluation matrix. The matrix is a tool for the operationalization of the evaluation questions, but it cannot be a substitute for the theoretical and methodological approach guiding the evaluation, which should be reflected clearly in its own section of the proposal.
It is the evaluation team’s responsibility to present in its proposal an appropriate theoretical and methodological framework for the purposes, objectives and utility of the evaluation (adjusted to the time and resources available for carrying this out), as well as a coherent approach that makes possible to operationalize in the evaluation matrix the evaluation’s objectives, levels of analysis and questions.
A preliminary evaluability analysis should be included, indicating which are the main a priori /enabling factors and limitations for satisfying the evaluation objectives for this evaluation, and answering the questions set forth.
In its proposal, the evaluation team will ensure: the complementarity and diversity of methods and information sources, making clear the existing limitations and specifying how and to what point the analysis is going to include the cross-cutting approaches such as right to health, gender equality and inclusion of diversity.
The evaluation will apply a mixed methods approach, engaging with key stakeholders of the project at all levels during the design, field work, validation and reporting stages. At least preliminary work session with Reference Group, desk review, interviews, discussion groups/surveys, observation and testimonies should be included. The data from these sources will be triangulated to increase the validity and rigor of the evaluation findings.
Presentation of the preliminary findings by the evaluators before the key stakeholders in a meeting hosted in the embassy of Spain at the end of the field work, to discuss and refine the findings and fill information gaps.
Reference group: AECID program manager, FCSAI project coordinator and/or specialist, MoH focal point
Primary Stakeholders: Heads of Directorates and Units at MoH – national and 3 regions level – directly involved in the implementation of activities; MoH Directors, directors of Clinics and health workers in clinics of the 3 governorates; project focal points for RHAS, JBCP, IFH and Our Step Association; Field coordinators, focal points in schools, volunteers, CBOs members, champions and NCD club members for JBCP, IFH and RHAS; sample of beneficiaries for Our Step, RHAS-HCC, JBCP and IFH. Also AECID in Jordan, Delegation of the European Union in Jordan and FCSAI.
Indirect interest (to be consulted): WHO Jordan Office, Royal Medical Services and Ministry of Education, at least.
The evaluation should have the following phases:
1) Preparatory activities and Desk Review. (1 week)
In this phase, the team must:
Phase I deliverable: Inception Report
The report is to contain the final evaluation design, which must include: i) a description of the evaluation object; ii) the intervention’s rationale (reconstructed if it were not expressly formulated); iii) the conceptual analytical framework; iv) a methodological proposal, including a brief evaluability analysis (describing the limitations found or foreseen and justifying, when applicable, any changes with regard to the initial plan), specification of the techniques and data collection tools, and a detailed analysis plan; v) the updated work plan; and vi) a definitive evaluation matrix.
2) Field work. (3 weeks)
Application of primary and secondary data collection tools according to what was set forth in the Inception Report.
Phase II deliverables: Presentations at the beginning and end of the fieldwork
The fieldwork will begin by briefing the Reference Group. Other stakeholders may also be invited to this briefing. Once the fieldwork phase has finished, the evaluation team will make a presentation that will include the activities carried out, and it will present very preliminary findings for discussion with the main stakeholders.
3) Analysis and interpretation of the information. (3 weeks)
In this phase, the team must:
Phase III deliverables: Final Report
The final report itself (maximum 70 pages without appendices), which must respond to the evaluation objectives and answer the questions set forth. The conclusions and recommendations in the report must stem from the findings.[1]
Final report structure should be as follows:
0. Executive summary (max. 5 pages)
1. Introduction
2. Summary description of the intervention
3. Methodology
3.1. Methodology and applied techniques
3.2. Limitations
4. Analysis and interpretation
5. Conclusions
6. Recommendations
7. Annexes, which will include: • Proposed methodology, techniques and sources used to collect the information: Documentary review, interviews, list of informants, interview scripts, transcripts and notes (if merited), and any other information that is collected and analyzed. • Claims and comments from different actors to the draft report if deemed appropriate.
Unless expressly indicated otherwise, all deliverables shall be in English. All deliverables shall be presented in Word and PDF, to guarantee their subsequent publication under adequate conditions, in whatever formats are considered appropriate. The final report shall include photographs, maps, tables, infographics and other visual resources to make the report more user-friendly and easily understood.
7.EVALUATION PRINCIPLES, AUTHORSHIP AND PUBLICATION
The evaluation shall follow the OECD DAC Quality Standards for Development Evaluation. Throughout the evaluation process, the evaluation team must respect the principles published in the Spanish Cooperation website.
Regarding authorship, without prejudice to recognition of the evaluation team’s moral rights, AECID is to be responsible for designing the layout, printing and publishing the documents.
The contract shall comprise at least a presentation of the results of the evaluation by the evaluation team. The evaluation team may also be required to participate in other activities involving presentation and delivery of results; in such cases, the costs that may be incurred by said activities are to be paid independently.
The evaluation team is to comprise at least 1 person. It is recommended that there be an appropriate balance between men and women, and that it includes local professionals.
The team coordinator must have a university degree and specialized training in evaluation or social research, and at least 3 years’ experience in carrying out evaluations. At least one of the members of the team or the individual evaluators shall be required to have experience in development or evaluation of projects, strategies or public policies in the health sector.
As a whole, the evaluation team must prove to have:
The technical proposal is to include the tasks to be carried out and the period that each professional will dedicate to the evaluation, as well as their formal commitment to being a member of the evaluation team during the validity of the contract. Any change to the makeup of the evaluation team must be previously agreed upon with the Reference Group.
Open to Companies and Individual Consultants with the capacity to provide this kind of service.
Bids are to be submitted by email to Mr. Francesc Vila, AECID´s Programme Manager (trough [email protected] and [email protected] ) before 23:59, 31th of May 2023.
The following documentation must be included:
The maximum budget for this evaluation is 14.800 EUR. The price shall be paid in three instalments, following the validation of the corresponding deliverables (adjust on a case-by-case basis):
The criteria for bids’ appraisal are the following:
Declaration of absence of conflict of interest (companies)
Mr/Ms …………………………………………………….., with ID no. ……………………………, in representation of the company …………………, with Tax ID no. ……………………, contracted to evaluate .…………………………………………………………………… starting on the date ………. ………………… 202…
I hereby declare, under my responsibility, that the company I represent is free from any conflict of interest that could affect the impartiality of the evaluation, and, specifically:
That the company I represent shall carry out the evaluation in a fully independent manner, without any type of pressure or influence.
That the company I represent has not participated in a relevant manner in the design or implementation of the object of this evaluation.
That the company I represent shall clearly state the existence of beneficiaries or partners of the evaluated interventions with which it has engaged in professional relations during the two years prior to this evaluation.
That the company I represent shall not establish contractual relations with the principal managers of the evaluated intervention to carry out design or implementation activities directly linked to the evaluation object for a minimum period of six months after the end of the evaluation.
That the company I represent shall refuse to obtain any advantage, either financial or in kind, constituting an illegal practice or corruption, as an incentive or reward relating to the evaluation object.
That the company I represent shall immediately inform the Evaluation Division of any other situation that may constitute a conflict of interest or that could lead to a conflict of interest.
And in witness whereof for the appropriate purposes, I hereby issue and sign the present declaration at ……………….., on ……… ……………………… 20…
Signature:
Declaration of absence of conflict of interest (individual)
Mr/Ms …………………………………………………….., with ID no. ……………………………, acting in my own name and right and as an external evaluator participating in the evaluation of ………………………………………………………………
I hereby declare, under my responsibility that I am free from any conflict of interest of a financial nature or relating to political affinities or ties of family or friendship which could affect my impartial performance of this evaluation, and to this end I undertake to:
And in witness whereof for the appropriate purposes, I hereby issue and sign the present declaration at ……………….., on ……… ……………………… 20…
Signature:
EVALUATION CRITERIA (WHEN APPLICABLE) |
EVALUATION QUESTIONS |
SUBQUESTIONS AND CLARIFICATIONS |
INDICATORS |
TECHNIQUES AND SOURCES |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[1] Whenever possible, amounts in currency should be stated in euros, regardless of whether other commonly-used currencies are also included in the evaluated intervention.
Activity Name: Improve Quality and Quantity of recyclable materials SOW No. and Title: Preparation of a QA/QC guideline for the...
Apply For This JobJob vacancy: Sales Agent, MetLife in Jordan We are looking for creative and driven salespeople to be part of a...
Apply For This JobMain Function: Recording and documenting all information and transactions and following up on all procedures related to work through the...
Apply For This JobTEAM/PROGRAMME: PDQ Technical Team External representation and communications: Level 2 -Regular contacts require the exchange of basic factual information to...
Apply For This JobInternational Orthodox Christian Charities Position Code (IOCC-FC-2023-02) International Orthodox Christian Charities (IOCC) is an international non-governmental development and relief...
Apply For This Job1. Purpose of the Position: Reporting to, guided and directed by the MEAL Manager, the MEAL Officer will be responsible...
Apply For This Job