In 2015, with a budget just creeping into six figures, my NGO provided psychological, socioeconomic and educational support to more than 150 families. We could have had more of an impact by splitting the budget among the families and leaving them to it.
To quote Duncan Green, strategic adviser at Oxfam GB, “We in the international NGO community are all too aware of our need to up our game in both understanding and demonstrating the impact – or lack thereof – of the various things we do”. To state the obvious, impact is important. In fact, the Center for Global Development’s Evaluation Gap Working Group was established more than a decade ago to “address the problem of the lack of knowledge about the effectiveness of social programmes in low and middle-income countries”. Unfortunately, it is not the responsibility of the CGD to ensure the effectiveness of such programmes. That responsibility lies with individual NGOs and it is for this reason that programmes can easily fall short of their potential.
Needless to say, endless evaluation is an impossible idealism. Funding is tight so why should NGOs invest precious revenue in costly impact evaluation when nobody is paying attention and flimsy statistics can be passed off as watertight findings? Eloquent websites and lengthy annual reports allude to responsible practices but often offer little substance. Many NGOs state a focus on “breaking the cycle of poverty” and report “a lasting impact” on families’ lives without any evidence to back up such claims.
Other NGOs may assert that X% of families have crossed the poverty line over the past couple of years. Often these figures are based on passing the World Bank’s $1.90-a-day threshold or using tools such as the Progress Out of Poverty index (PPI). However, neither NGOs, nor the families they serve, exist in a vacuum. Yes, these families may no longer technically be impoverished, but attributing an extra dollar a day to your NGO’s recent microloan or an extra couple of points on the PPI to your workshops on family planning is inaccurate and unprofessional. Correlation is not causality.
Unfortunately, my own NGO is among those that do not recognise this. We pile huge amount of resources into measuring the attendance of families without evaluating the quality of the programmes they are attending. We collect relatively large amounts of data on participating families but this information is not used to inform the decisions we make and is eventually lost in a labyrinth of Excel databases. There is a difference between censusing the local population and measuring the impact of specific programmes.
Frustratingly, while our own evaluation gap grows, more focus is put on the experience of volunteers. Interestingly, they are surveyed at the end of their placement, unlike the families who leave our programme. Positive feedback provides a flashy quote to excite future volunteers, while negative feedback is largely ignored.
There is no substitute for a robust impact evaluation of your programme. This does not have to be done annually; mixed-methodology research, where you compare a group working with your NGO with a group that is not, can be achieved in a relatively small time frame. So why is there such a scarcity of such practices?
As I mentioned earlier, money is often a factor. Another is a chronic lack of regulation. Any NGO will be governed by an executive board who, collectively, will determine that organisation’s direction and policies. These will, in turn, be overseen by the Charities Commission for England and Wales (if your charity is based there) – a body that is supposed to regulate 165,000 charities on an annual budget of £22m. Thus, if you are one of the 160,000 and counting NGOs who are not “regulated” in any given year, it’s down to the board and the CEO. My NGO’s board does not consist of one national from the country we operate in. Some of them have never travelled to this country, the rest almost never visit and one of them is my CEO’s mate. Sadly, this is part of a bigger trend; studies are scarce (deciding to investigate why your board is too male or too European is not popular) but NGO boards display a worrying lack of transparency. Even among the best, these practices are commonplace – our NGO was ranked highly in our continent by the Top 500 NGOs index in 2015. Though worrying, this lack of self-evaluation is far from surprising. Questioning the impact of your NGO is merely inconveniencing yourself; better to sit on a pedestal, recycle some mindless statistics and keep the volunteers queueing up and paying in.
None of this is to question the goodwill of many NGOs but goodwill is not, nor will ever be, sufficient. NGOs that are unable or unwilling to provide strong evidence of the impact they are having are, at best, a considerable waste of time and money. What seems a lot more certain is that many NGOs are continuing to provide noble career paths and selfless volunteer placements for the more fortunate, while simultaneously servicing the local population with untested and meagre programmes.
Any change of approach will have to originate from within the international NGO community, otherwise for as long as this status quo rumbles on, so will the ugly questions of neo-colonialism and white privilege that often circulate around this kind of work.
—The Guardian
