Dr Anastasia Aladysheva :
(From previous issue)
What is the most effective way to provide the nutrition assistance given the particular country context? If providing assistance to the head of household leads to the optimal distribution of resources across all household members, this is the optimal solution. If, however, in contexts where the head of household is nearly always male, it leads to an unequal distribution of resources by limiting access by young children and women, then providing it to mothers may benefit the disadvantaged members of the household.
Providing assistance directly to a woman rather than a man, however, could put a woman at risk in the most male-dominated societies. Famine is not always about there not being enough food. Providing food vouchers or cash in the absence of functioning food markets, however, will not be an effective solution. An impact evaluation helps to answer some of these questions.
What is the magnitude of the impact of aid? To be able to understand the process of how joint humanitarian efforts affect selected nutritional and health outcomes, nutritional, health and other data should be collected, ideally both before and after the assistance is distributed at the individual, household and community levels.
What are the impacts of aid provided by a particular agency? To answer this question, it is important to define an identification strategy that is able to attribute impact to a particular intervention. First, it is desirable to have a comparable ‘sample’ to which it was not possible to provide an intervention, or which received an intervention at a later stage, to compare with the intervention group that received assistance.
Having a control group helps to eliminate parallel ‘shocks’ that may influence both groups at the same time and which are not due to the intervention. To guarantee fair allocation and comparability, both samples must be selected on a random basis before the intervention-in a randomized control trial.
In a humanitarian context, however, it is often the case that large areas or a large number of the population are affected (a high co-variability problem), or there would be ethical considerations around the agency delaying assistance any longer than necessary. Therefore, withholding assistance from a certain group of the population could not be considered. However, when resources are limited, or the intervention has potential side-effects as in public health programmes, random selection could still be considered. The impact is the difference between the intervention group and the comparison group before and after the programme.
Another solution would be to start evaluating the programme after the intervention. In such cases, the evaluation team must rely on existing data available on the community before the disaster.
How sustainable are these impacts? In this case, researchers return to collect additional waves of data at a later stage (medium to long term). The nature of the intervention defines the timeline of the evaluation. In the case of nutritional programmes, the research team may come back after six months, one year or even longer, depending on the scale of the programme and the availability of resources.
What amount of assistance is needed and at what frequency should it be delivered? An impact evaluation team should perform rigorous statistical analyses using the data collected. It is therefore important to ensure that the data contains full information on the outcome indicators that the implementation team was aiming to improve, such as health indicators, the dietary diversity of all household members, and so on.
Questions specific to the intervention should be formulated – and if possible piloted in advance – in a such a way that it is possible to learn detailed lessons about the impact of the programme.
Did certain groups within the affected population become better off than others? If so, in what ways and to what extent were their livelihoods improved? The most vulnerable are usually hit the hardest by disasters. Together with the implementation team, the impact evaluation team should help to identify the intervention target group. During humanitarian emergencies, it is important that data on displacement and an existing census or similar data are accessible to the research team.
Were there any unintended or negative consequences? When providing household-targeted interventions, it is essential to understand how resources are redistributed among all members of the households and whether certain groups have restricted access. Therefore, even though this requires more financial resources, surveying all household members is essential. Could aid be delivered in a more cost-effective way? Impact evaluations using randomized control trials can and have helped to identify the type of assistance-providing in-kind food, food vouchers or cash-that is most efficient or appropriate-in terms of both cost and effectiveness-for the target population and the conditions on the ground.
The implementation and research team should identify the needs of the population and the context in which the intervention will take place before the intervention starts, using interviews during field visits and desk research.
To summarize, all the nuances of the intervention must be considered in order to be able to design an impact evaluation. Therefore, the implementation and impact evaluation teams should synergize to guarantee that all the available information is accessible to both.
The main advantage of a rigorous impact evaluation is that it contains a set of components that have been tested in many development programmes and that in combination can help to tease out unbiased impacts.
Some methodologies-such as randomized control trials-are flexible enough to be adjusted to the humanitarian context and complement the existing monitoring and evaluation methodologies of humanitarian agencies. The key recommendations to humanitarian agencies are therefore:
o Learn about where disasters may unfold and where assistance may be most required. Many natural disasters happen in the same geographic areas. Collect comprehensive data on the populations and local environments likely to be affected that will be useful in times of disaster.
o Prepare and design impact evaluations in advance that are specific to the types of intervention the agency conducts.
o As the climate changes, it is highly likely that the number and scale of disasters will grow in the coming years. The need for humanitarian assistance is therefore also likely to increase. This creates an urgency to ensure that humanitarian aid is spent in the most effective and efficient way.
(Dr Anastasia Aladysheva is a Senior Researcher with the Peace and Development Programme at the Stockholm International Peace Research Institute (SIPRI)
(From previous issue)
What is the most effective way to provide the nutrition assistance given the particular country context? If providing assistance to the head of household leads to the optimal distribution of resources across all household members, this is the optimal solution. If, however, in contexts where the head of household is nearly always male, it leads to an unequal distribution of resources by limiting access by young children and women, then providing it to mothers may benefit the disadvantaged members of the household.
Providing assistance directly to a woman rather than a man, however, could put a woman at risk in the most male-dominated societies. Famine is not always about there not being enough food. Providing food vouchers or cash in the absence of functioning food markets, however, will not be an effective solution. An impact evaluation helps to answer some of these questions.
What is the magnitude of the impact of aid? To be able to understand the process of how joint humanitarian efforts affect selected nutritional and health outcomes, nutritional, health and other data should be collected, ideally both before and after the assistance is distributed at the individual, household and community levels.
What are the impacts of aid provided by a particular agency? To answer this question, it is important to define an identification strategy that is able to attribute impact to a particular intervention. First, it is desirable to have a comparable ‘sample’ to which it was not possible to provide an intervention, or which received an intervention at a later stage, to compare with the intervention group that received assistance.
Having a control group helps to eliminate parallel ‘shocks’ that may influence both groups at the same time and which are not due to the intervention. To guarantee fair allocation and comparability, both samples must be selected on a random basis before the intervention-in a randomized control trial.
In a humanitarian context, however, it is often the case that large areas or a large number of the population are affected (a high co-variability problem), or there would be ethical considerations around the agency delaying assistance any longer than necessary. Therefore, withholding assistance from a certain group of the population could not be considered. However, when resources are limited, or the intervention has potential side-effects as in public health programmes, random selection could still be considered. The impact is the difference between the intervention group and the comparison group before and after the programme.
Another solution would be to start evaluating the programme after the intervention. In such cases, the evaluation team must rely on existing data available on the community before the disaster.
How sustainable are these impacts? In this case, researchers return to collect additional waves of data at a later stage (medium to long term). The nature of the intervention defines the timeline of the evaluation. In the case of nutritional programmes, the research team may come back after six months, one year or even longer, depending on the scale of the programme and the availability of resources.
What amount of assistance is needed and at what frequency should it be delivered? An impact evaluation team should perform rigorous statistical analyses using the data collected. It is therefore important to ensure that the data contains full information on the outcome indicators that the implementation team was aiming to improve, such as health indicators, the dietary diversity of all household members, and so on.
Questions specific to the intervention should be formulated – and if possible piloted in advance – in a such a way that it is possible to learn detailed lessons about the impact of the programme.
Did certain groups within the affected population become better off than others? If so, in what ways and to what extent were their livelihoods improved? The most vulnerable are usually hit the hardest by disasters. Together with the implementation team, the impact evaluation team should help to identify the intervention target group. During humanitarian emergencies, it is important that data on displacement and an existing census or similar data are accessible to the research team.
Were there any unintended or negative consequences? When providing household-targeted interventions, it is essential to understand how resources are redistributed among all members of the households and whether certain groups have restricted access. Therefore, even though this requires more financial resources, surveying all household members is essential. Could aid be delivered in a more cost-effective way? Impact evaluations using randomized control trials can and have helped to identify the type of assistance-providing in-kind food, food vouchers or cash-that is most efficient or appropriate-in terms of both cost and effectiveness-for the target population and the conditions on the ground.
The implementation and research team should identify the needs of the population and the context in which the intervention will take place before the intervention starts, using interviews during field visits and desk research.
To summarize, all the nuances of the intervention must be considered in order to be able to design an impact evaluation. Therefore, the implementation and impact evaluation teams should synergize to guarantee that all the available information is accessible to both.
The main advantage of a rigorous impact evaluation is that it contains a set of components that have been tested in many development programmes and that in combination can help to tease out unbiased impacts.
Some methodologies-such as randomized control trials-are flexible enough to be adjusted to the humanitarian context and complement the existing monitoring and evaluation methodologies of humanitarian agencies. The key recommendations to humanitarian agencies are therefore:
o Learn about where disasters may unfold and where assistance may be most required. Many natural disasters happen in the same geographic areas. Collect comprehensive data on the populations and local environments likely to be affected that will be useful in times of disaster.
o Prepare and design impact evaluations in advance that are specific to the types of intervention the agency conducts.
o As the climate changes, it is highly likely that the number and scale of disasters will grow in the coming years. The need for humanitarian assistance is therefore also likely to increase. This creates an urgency to ensure that humanitarian aid is spent in the most effective and efficient way.
(Dr Anastasia Aladysheva is a Senior Researcher with the Peace and Development Programme at the Stockholm International Peace Research Institute (SIPRI)
