There are three types of data in international development projects; the good, the bad and the ugly.

Ugly is when data is collected for the sake of collecting data. This means the people collecting data have pre-decided what it is going to look like and the people demanding that data are disinterested in its quality. This kind of data is often collected under a tree, or in a community centre, while sipping on cups of tea. The data is made to fit the agenda of the project. Hence if the idea is to construct a water filter, data will be made to show that people in the project area are suffering endlessly from diarrhoea, cholera and skin rashes. Ugly. Because it is fictional, cannot be used for any good purpose, and looks really bad when shared. Hence, many people don’t share their data because they know it is ugly. If you are looking at ugly data, I suggest you use your Joo Junta 200 Super-Chromatic Peril Sensitive sunglasses.

Bad data is data in existential crisis. It’s collected with honest intentions, but nobody knows why it is being collected. The data reflects lack of clarity, which then goes on to vex people who are analysing it. This data may make some sense, maybe somewhat useful, but doesn’t make for great analysis. I generally like to stay away from bad data – I find it to be very irritating. Like Marvin the Paranoid Android it continuously whines “Life? Don’t talk to me about life.”

Good data generally gives you goose bumps, in a nice kind of way. It was collected with honest intentions, the people asking the questions knew exactly why, the format they used had no leading questions, good proxy indicators were used, and conclusions were arrived at by looking at the data on the spread sheet and not pre-decided during data collection by the data collector.

Good data is rare – like a sunny day in Amsterdam – and you definitely want to make the most of it. However, to find it one needs to be focussed like a pig looking for truffles.

I found it recently, in Nepal with the Nepal WASH Alliance.

Last year the Nepal and Bangladesh WASH Alliance consortia took the lead with using Akvo FLOW to measure the outcomes of their interventions to improve access to clean water. The survey formats were decided by the WASH Alliance in each country. 2,800 households’ were surveyed in Nepal to understand how the projects were coming along. When I started analysing the data, I got an interesting set of results. It seemed a large number of households had a functional drinking water source.

Great stuff, right?

But the data doesn’t really speak to us. It doesn’t tell us how functionality was deduced; whether it was functional only on the day of the visit or functional throughout the year, and if it’s adequate for everyone in the household. For example, if I ask someone, “Do you get water in your tap?” and he/she says “Yes”, it’s not enough, because the water may come in a trickle and take ages to fill a bucket. Then that water point is providing water but not “adequate” water. Plus, is the current supply sustainable? Is there an institution managing it? Are people collecting tariffs to maintain the source?

Essentially, your data is as good as the questions you ask. And more importantly, data collecting organisations need to understand that the questions they ask should help them to implement their project better. Unless their data talks to them, they will not be interested in its quality.

Top: Nepalese masks, Kathmandu. Photo by Amitangshu Acharya.
Above and below: up to the minute visualisations of data are viewable in the programme’s Akvo FLOW dashboard. As this is household data, it is not made public, however the aggregated data will be published in time.

Because we could see the analysed data on FLOW as soon as data collection was completed, learning from it was applied immediately. The outcome-monitoring format in Nepal was redesigned with the active participation of the monitoring and evaluation lead person at the Nepal WASH Alliance Jurrie de Hart, WASH Alliance representatives and myself. This is now being used for developing a new baseline for Nepal WASH Alliance, on the basis of which outcomes will be tracked from now on. Data collection has begun, and we are getting a richer set of data. It’s pure joy watching it on the FLOW dashboard:
 

The progress in Nepal is significant. With a new and more robust questionnaire in place (and we can keep improving it) we will get good data, which can be shared openly (at an aggregated level) and also provided to the Government of Nepal. In the process the WASH Alliance can set new benchmarks in Nepal’s water sector through transparent reporting and data sharing.

Similarly, my Akvo colleague Rajashi and I are now working with the Bangladesh WASH Alliance, so that future monitoring is based on good quality data. The diversity of projects within the WASH Alliance is dazzling. Monitoring such a wide range of projects means that partners need customised monitoring formats based on the nature of their activities, which should then be aligned at both the national level and with Dutch WASH Alliance (DWA) indicators. They also need to know how to sample the households/assets they intend to monitor and how to prepare monitoring schedules. And finally the analysed data should not only feed into DWA reporting at a country and global level, but also – and more importantly – help the partners understand the progress of their projects better and design strategies accordingly.

We have worked with one partner to redesign their monitoring format. As an advocacy organisation, they needed a different set of monitoring questions to measure their progress. We wish to extend the same support to all other partners who need it.

Lastly, the outcome monitoring exercise in Nepal and Bangladesh started in November and ended in December 2013. And by March 2014 we have already redesigned the questionnaire based on the results and redeployed it for baseline data collection in Nepal. All this has happened within five months. If we were using conventional pen and paper methods, it would have taken much longer. Using mobile tools has accelerated access to and analysis of data.

At the end of the day, good data collection is not just about using smartphones; it’s more about smart questions. When the two come together, magic happens. We are booking the front seats for seeing this happen on the dashboard.

Amitangshu Acharya is the manager of Akvo’s Asia hub.