Relevant Experience: Survey and Evaluations

 Job Description:

Affirmation   

All the contents provided in Terms of Reference (TORs) are only limited to serve the purpose stated herein. Contracting Organization prohibits use of information/data for any other purpose. Any re-production of these TORs is strictly not allowed. Contracting Organization does not accept responsibility for any interpretation of these TORs beyond scope of the assignment.  

 Endline Survey Summary

About Us

We aim to contribute to a just and prosperous Pakistan where all citizens enjoy equal rights and opportunities. Together with Pakistani civil society organizations, national partners, academic institutions, and the Government Agencies, we work to implement long-term development and humanitarian programmes in:

• Cultural Cohesion, Understanding & Diversity
• Gender Equality, Protection & Respect
• Climate Resilient Water, Sanitation and Hygiene (CR WASH)

In pursuit of global and country strategic objectives, we have initiated two 5-year long programmes in 2020 in seven districts of Khyber Pakhtunkhwa, Punjab, and Sindh. These are:

 i) Faith in Action for Sustainable Climate Resilience

ii) Joint Social Action for Inclusive Society. 

Both projects are in the last year of implementation and registered progress against activities, interventions, and results. The endline survey will provide evidence to measure the progress against predefined indicators from their baseline values and/or lacking thereof for three thematic foci of both projects. The programmes/projects details are as follows:  

Faith in Action for Sustainable Climate Resilience 

According to Global Climate Risk Index, Pakistan ranks 7th as the most vulnerable country to climate change and witnessed catastrophic natural disasters induced by climatic change and was least prepared to address the climate change crises. According to IMF, Pakistan ranks 3rd amongst countries at risk of acute water scarcity. These conditions, combined with weak infrastructure for the provision of adequate water and sanitation services, health facilities, and lean livelihood opportunities, have increased the multi-dimensional vulnerabilities of the population of targeted districts. Lack of awareness among communities about climate change-related risks and issues, haphazard planning, and lack of preparedness make the problem more complicated. The Strengthening Civil Society aims to have inclusive, resilient, and empowered communities to lead climate action for discernible improvements in the quality of their lives. The programme is being implemented in three districts (Mardan, Peshawar, and Swat) in Khyber Pakhtunkhwa and two districts (Sanghar and Umerkot) in Sindh. The programme integrates all three thematic foci of the organization to achieve the objective through improved health and nutrition of most vulnerable communities and women-led households to cope with recurrent natural disasters and resource depletion. The mobilization and strengthening of local and national civil society organizations and stakeholders including faith actors to influence locally accepted and sustained awareness and advocacy drive on climate change are integral approaches of the programme. 

 Joint Social Action for Inclusive Society 

 The overall goal of this project is to “support joint social action for an inclusive society. The organization and partners will focus interventions at two levels. First, civil society and faith actors collaborate with duty bearers (relevant Government Departments) to promote cultural cohesion among diverse communities and build alternative narratives for equal opportunities. Second, local communities prevent and transform disputes and protect their basic rights. The organization will strengthen partners technical capacities, while partners will engage other civil society and faith actors, and duty bearers for effective project implementation that safeguards its relevance and sustainability.   

The project will leverage faith actors as key change agents because they are rooted in and have high level of legitimacy in community and outreach to the most vulnerable. The target areas (Umerkot in Sindh and Lahore and Faisalabad in Punjab) are selected due to concentration of vulnerable economically disadvantaged segments of the society with history of social problems and poverty due to lack of awareness and equal opportunities. At community level, the project will target local communities (women, youth, religious leaders, minority representatives, journalists, and teachers), while for advocacy it will target key duty bearers. The project will contribute to vision of organization’s 2020-2024 country strategy: A just and prosperous Pakistan where all citizens enjoy equal rights and opportunities.

Purpose of the Endline Survey, audience, and use

The purpose of this study is to assess progress made in projects for the output and outcome level indicators corresponding to benchmarks and baseline values established on the bases of baseline conducted in 2020. Also, highlight any important learning on the change in context that contributed to the achievement of the planned progress or lacking thereof. Providing recommendations for course correction for future planning of projects interventions.

The findings of the survey/study will be used by the commissioning organization for the purpose of accountability and reporting to projects sponsors, organization headquarter, and partners. The thematic programmes will utilize the findings of the study/survey to align future planning and programming discourse. The abridged findings of the survey will also be shared with the other stakeholders including government line departments, CSOs, and community members.

Specifically, the endline survey will serve the following objectives: 

• Assess/measure the endline values of key global outcome indicators specified in the projects’ Results Frameworks –disaggregated by sex, age, and disability status as applicable— against the baseline values and benchmarks set forth.  
• Capture human interest stories or case studies to reflect positive and/or negative contribution towards projects’ objectives and goals.

 Geographic locations:

The geographical scope for the survey/study will be the seven districts the projects are being implemented in. The table summarizes the locations specific to projects.

 Indicators to be measured:

Methodological approach

The commissioning organization seeks potential consultant/consulting firm to propose the comprehensive methodology to conduct this study/survey to measure the results of organization’s five-year project (2020-2024) supported project. However, the endline survey/study will utilise both qualitative and quantitative methods to collect information, ensuring in-depth analysis of status of outcome and output indicators. The information collection methods will include documentation reviews including the proposals, results framework, national and local development plans and strategies, and reports (progress, monitoring, and baseline); interviews of individuals participated in project activities; focus group discussions; and key informant interviews with relevant partners and stakeholders.  To greater possible extent, the consultant should ensure the survey is conducted in a participatory manner and ensure that all the relevant stakeholders should be meaningfully and fully involved in the process. The commissioning organisation will provide the methodologies for measuring the endline of all the global outcome indicators. For recoding case studies consultant should propose her/his methodological approach.  

The consultant is expected to coordinate planning, enumerators trainings for accuracy in information collection, field tool testing, and analytical findings as per organization’s reporting standards/criteria. The consultant will propose a specific methodology in the inception report to the contracting authority for approval before proceeding with the field work. The inception report must provide details on the sampling strategy (sample frame, size, and technique), data collection and analysis plan/methods, and quality assurance measures along the processes. Organizational minimum standard to sampling will be shared with the consultant along with consultancy agreement. The information collection and analysis must include faith diversity, gender, age and disability disaggregation.  The methodology proposed for the survey must provide information that is sufficiently representative, detailed and accurate. 

The survey must account ethical considerations and standards to information collection, analysis and dissemination of findings. The consultant shall commit to protect the rights and confidentiality of informants and abide by “do no harm” principles. The consultant shall ensure the compliance to General Data Protection Regulation (GDPR) throughout all the phase of the assignment. Information collected in the course of the assignment will be used for the purpose of the assignment only and handed over to the organization once the assignment has been completed.

Roles and Responsibilities

Responsibilities of Commissioning Organization

• Brief/orient the consultant on the programmes and projects.
• Provide global indicators guidelines and methodologies.
• Review, provide feedback and approve the survey/study tools, methodology, and reports (inception, draft, and final reports)
• Brief and orient stakeholders about the purpose of the survey
• Extend necessary facilitation to ensure coordination among partners, stakeholders, and consultant.
• Provide necessary support to the consultant to ensure timely completion and compliance with the international survey standards.
• Assist in organizing meetings with stakeholders and partners.
• Prepare and disburse payments for the consultant upon completion of the deliverables articulated in the ToRs.  
• Provide necessary documentation to the consultant, workout a programme and coordinate throughout the fieldwork.
• Provide clearance to assignment deliverables that meet quality standards
• Review and approve inception and final reports of assignments including data collection tools.
• Process and approve payment as per the payment schedule mentioned in the contract

Partners’ Responsibilities

For the endline survey, partners role includes:

• Provision of partner level relevant documents and project participants list as necessary for desk review.
• Facilitate consultants in coordinating and establishing linkages of consultants with communities and key stakeholders for the purpose of data collection.
• Facilitate commissioning organizations to monitor field data collection processes and activities for the purpose of data quality assurance.

Consultant’s Responsibilities

Under the guidance of the commissioning organisation’s focal point, the role of the consultant/consulting firm is to:

• Hold an inception meeting with the commissioning organization.
• Desk review of relevant programme and projects documents and reports amid proposals, theory of change, assessments, project financial plan, monitoring reports, and baseline reports.
• Prepare and submit to the commissioning organization the proposal and inception report including employed methodology, survey tools/questionnaires, work plans, and schedule for both qualitative and quantitative aspects of the study for review, feedback, and approval by commissioning organization.
• Establish contacts with all the relevant stakeholders in target communities/districts.
• Design and pilot data collection tools.
• Capacity building of the data collection team.
• Lead, implement, and supervise the primary data collection.
• Field visits to selected communities; interview and/or FGDs with partner organizations, faith actors, teachers, students, community members, and other stakeholders in accordance with approved methodology.
• Conduct meeting/workshop with commissioning organisation, partners and other stakeholders to debrief on and present findings and recommendations of the survey.
• Submission of draft survey report and finalize it based on the feedback from HO and CO of commissioning organisation. The Consultant is expected to submit a hard copy of final approved report and electronic copy along with the collected data in both hard and digital form.
• Submission of individual recommendation for commissioning organization and its partners, separate for both projects. 
• Handover all the data, both hard and soft data, produced under this survey and ensure compliance to GDPR, data protection protocols and Don No Harm Principles.

Expertise and Experience

To be considered for this role the consultant must have:

• An advance degree in statistics, research, monitoring and evaluation, development studies or related discipline.
•  At least 10-15 years of relevant professional experience in surveys/research designs, sampling strategies, and methods of data collection, analysis, and quality control.
• Experience in conducted quantitative and qualitative surveys/research/studies, specifically in Pakistan.
• Extensive knowledge in development sector, community development and social cohesion.
• Good knowledge and understanding of socio-political, cultural context and diversity landscape of Pakistan.
• Knowledge of conflict sensitivity and gender mainstreaming.
• Documented good understanding of vulnerable/marginalized communities of society (including faith communities); and
• Have good skills on data collection, analysis, communication, writing and presentation.
• The applicant preferably affiliated with reputable research institution or consultancy firm (from Pakistan).

Teams Composition

The applicant (consultant/consulting firm) can propose a structure that will work effectively to execute this Terms of Reference. The key suggested positions and qualifications include:

Principal researcher/Team Leader: A principal researcher should be at the senior level and is required to have at least 10 years relevant experience with at least a Master’s Degree, and preferably PhD, in a relevant area. As the leader for this assignment, the position will hold a primary responsibility for the technical quality, progress, and deliverables of the MTS. The position needs to have excellent skills primarily in both quantitative and qualitative research, data processing and analysis. The position needs to have excellent English proficiency. A principal researcher with a very good local knowledge of the study areas and people there is a strong plus.

Cultural diversity and inclusion specialist: The position should have at least 7 years relevant experience with a master’s degree in community development, conflict transformation, or other related social sciences (e.g., anthropology or sociology). The position should have practical experience in guiding and leading research activities in areas of cultural cohesion, understanding, and diversity. The position should be recognizant with the diversity spectrum of Pakistan, specifically the area of study. 

CRWASH specialist: This position should have at least 7 years relevant experience with an advance degree in engineering, health communications, public health management or other related discipline (e.g., environmental sciences). The position must have experience in working on technical assignments including KAP surveys, assessments, vulnerability and capacity assessment in perspectives of climate resilient WASH.

Database manager/data analyst: These positions should be experienced in setting up database and analysing quantitative and qualitative data using professional qualitative data analysis software, with skills in coding and synthesising quantitative and qualitative data based on the analytical framework of the survey protocol. The position needs a advance Degree in relevant assessment areas.

Gender and social inclusion specialist: At least a Master’ degree or equivalent in social sciences, gender/women study, or other development-related fields (e.g. development studies) and 7+ years of experience in the area of gender mainstreaming and women’s empowerment, preferably on projects in areas of gender inclusion, cultural cohesion, understanding, and diversity, and Climate Resilient WASH. The position should have experience in working on gender technical assignments, including gender analyses, assessments and survey research methodologies in particular for baseline/midline/endline surveys in the rural and urban setting.

Data collectors: This position should have relevant qualification and practical experience in data collection in the surveys/assessments.

Communication Lines/Structure

For this assignment the reporting point for consultant at commissioning organization is Head of Programme (HoP), however, for the coordination purposes the focal point will be assigned.

Deliverables

The key deliverables of the assignment are enlisted below.

• Inception Report: to be provided by the Consultant at the start of the assignment with detailed study design, dummy output tables and work plan based on information provided and discussions with the focal point and an outline of the endline report format.
• Endline Survey Instruments (Questionnaires, checklist of FGD, checklist of Key Informants, etc.
• Refined pre-testing questionnaires (local language if applicable)
• Training of Enumerators and Pilot
• Draft Endline Survey Report and raw data set (both hard & soft copies) for each project
• Workshop to share the summary of findings and recommendation
• Final Endline Survey Report (both hard & soft copies) after incorporating feedback

Timeline

The endline Survey report and findings will be expected to come within 50 days, anticipated to begin in November 2024 and end by 20^th December 2024. The tentative timeline and the key deliverables of the assignment are given in the following table.

• The consultant would submit Inception Report (max. 15 pages excluding annexes), presenting the methodology (tools, sampling etc.), framework of survey and timeline. The inception report will have to be approved by the commissioning organisation. For template refer Annex-2.

• The final report should not exceed 30 pages and shall cover:

-      Executive Summary (2 pages)

-      Overall review methods, their appropriateness and constraints faced, if any.

-      Clearly identified findings, conclusions, lessons learnt (30-35 pages)

-      Recommendations (1-2 pages)

-      Appropriate Annexes, i.e., TORs, data collection instruments, list of interviewees, raw collected data, dataset on Nvivo, SPSS, STATA, excel (as and attached file), and any auxiliary information germane to this survey.

Moreover, the organization’s relevant programme staff will be responsible to organize coordination between consultant and field-based partners and administrative arrangements (if any) ahead of time. If required, organization and its partners’ staff will ensure target communities in the field are prepared and ready to participate in interviews, FGDs etc. to collect required information from the field. 

Budget

The budget proposed for the assignment should cover all costs for remuneration of experts/staff including their travel, training, logistics, stationeries, and miscellaneous cost. The detailed budget, terms and conditions for the applicant will be separately spelt out in the respective contract.

Mode of Payment

Payments will be made in the following three instalments as per the schedule described in the assignment agreement. The consultant must be a tax filer. The commissioning organization will deduct the tax at source according to the country’s laws and regulations.

Logistics and other support

The consultant is responsible for all the logistic arrangements of his/her team necessary for the completion of this assignment.

Requirements of Proposal

• The applicant/consultant is expected to submit technical and financial proposal, separately. It must be ensured that financial information is not included or disclosed in technical proposal. The proposals should comprise of the following components:
• The consultant understanding of the Terms of Reference (ToRs).
• A clear and elaborated work plan.
• Organizational capacity statement and experience related to project evaluations and endline surveys.
• Curriculum Vitae for all the consultant and complementary staff. (CVs submitted should of staff that will be engaged in this assignment).
• Professional Reference of three organizations/employers; detail include Names, address, telephone numbers of referee organizations/employers.
• Copies of relevant work undertaken in last three years.
• Full names, post office box, telephone numbers, email address, physical address, and contact person of the consultancy.
• Financial plan for this assignment including schedule of payments.

Proposal Evaluation Process

The process to evaluate potential proposal comprises of three steps. The evaluation will have a maximum of 100 points composed of 80 and 20 points weightages for technical and financial proposal, respectively.

Step 1: Compliance Check

The Procurement Committee (PC) will carry out an examination of the administrative compliance of the technical proposal.

• NTN registration
• Tax Clearance/Tax Return Submission (latest FY2021-22)
• Letter of Application
• Sample work for the latest two assignments
• Two reference letters from the latest clients
• Technical and financial proposals

Step 2: Technical Assessment:

The PC will evaluate the technical proposal in accordance with the technical criteria as described below.

Relevant Experience = 15

Team composition/Personnel = 20

Conceptual framework for Survey/Study = 30

Assignment organization and delivery plan= 15

Step 3: Financial Assessment:

The PC will evaluate the financial proposal following the “value for money” approach. The proposal that can deliver the best balance between the high-quality implementation of deliverables and cost will score highly.

Financial proposals will be evaluated on a maximum financial score of 100. The lowest financial proposal receives a financial score of 100. Other proposals are allotted financial scores as follows: Financial score = (lowest Financial Proposal / Financial Proposal being evaluated) x 100

Timeline of Proposal Submission

EOI will be submitted on “[email protected]” no later than by CoB November 16, 2024.