Trading surveys for conversations with ODK

Jan 5, 2017

Andrew Bell is an Assistant Professor at the Department of Environmental Studies at New York University. He studies the use of surveys, experimental games, and choice experiments to inform the development of agent-based models of agricultural decision processes. In this guest post, Andrew describes work done with Patrick Ward, Mary Killilea, and Md. Ehsanul Haque Tamal developing "microtasks for micropayments" programs with Nafundi and ODK.

Could you remember how much you'd paid for groceries over the last few months if asked? Would it be any easier if you'd already spent hours answering questions about your job, your family, your spending, and your future plans?

It would be daunting, and yet it's not too different from how social research works in rural areas across the world. Survey efforts are costly endeavors, so that return visits to the same people are often far apart in time, and try to cover a lot of ground in one sitting. The diffusion of inexpensive smartphones across the rural landscape has the potential to change that, by allowing researchers to connect with respondents regularly, asking a few questions at a time along a week, month, or year, and in doing so capturing some of the details (and variation) that might otherwise get missed.

We recently published early findings from a pilot study among 480 farmers in Rangpur District in northern Bangladesh, in which we stepped out ahead of the smartphone frontier to examine how such a mode for rural engagement might take shape in the years to come. Participants were issued an Android handset from the mobile provider Banglalink (a Symphony Roar V25 with Android 4.4.2), and were able to earn small packets of mobile data, airtime, and credit toward ownership of the device by responding to short survey tasks in English and Bangla made available to them daily using Open Data Kit (ODK) forms designed by Nafundi. While most of the analysis is still in front of us, a few things jumped out from the early results.

We randomized how frequently participants would receive a given task (e.g., once a week, once a month, or once a season), and found that those who reported weekly, reported more. Those who reported income opportunities weekly listed more – and smaller – earnings sources, possibly showing an underbelly of diversified incomes that might be forgotten by the time a survey team rolled in. Those who reported weekly on school absences among children reported more of them, and importantly, peaks in illnesses showed up earlier in the data than among those recalling only monthly.

New users learning how to use the Android device. Photo courtesy of Md. Ehsanul Haque Tamaln.

We observed response rates over 80 percent for most of the tasks sent to respondents, including 'crowdsourcing' tasks, where those holding the handset were asked to flag down a friend, neighbor, or passerby and elicit their responses to questions, instead of responding themselves. We found the distribution of crowdsourced respondents to be significant step closer to a representative sample (using the indicator of years of education), suggesting that approaches like this might help to overcome sampling bias in the coming years when only the few (younger, affluent, and tech-savvy) have access to smartphones in rural areas.

Taken to scale in the years to come, a platform like this that connects researchers, development practitioners, and policymakers around the world directly with rural communities could revolutionize the way research is performed, or the way programs are monitored and evaluated. Research funds could be channeled directly into mobile payments, and the results of interventions observed and acted upon in near-real time. However, there are numerous challenges to overcome before that could become a reality. First among these is to streamline the process for rural farming households to engage directly in research so that it is as seamless for them as it is for many of us to order dinner or re-tweet.

ODK is designed to streamline the data collection experience for the trained enumerator. It makes the tasks of managing survey forms and aggregating data simple, and allows most survey items to be completed with a thumb or forefinger (and usually not both). However, while it greatly eases the job of the trained enumerator, it isn't necessarily designed to take his or her place. In our current pilot, and in future work moving forward, Nafundi is helping us to design a platform that could potentially eliminate the need for on-the-ground enumeration teams.

To start, we needed software to manage survey tasks: to schedule them for the appropriate time, alert participants when they were newly available or about to expire, and take them directly into the task they wanted to complete. Nafundi built a Android app for us, called Data Exchange, that did precisely this. Data Exchange reads header information from ODK forms preloaded to the device and displays only those that are meant to be available to the user, along with other basic information such as the value of the task (in minutes of airtime and MB of mobile data) and the expected difficulty. Next steps will develop the underlying engine of Data Exchange to make it easier for the results of one survey task to inform another, such that the long-term engagement becomes more of a meaningful conversation for the respondent, and a more tailored data collection exercise for the researcher.

Screenshots from the Data Exchange app's notifications, user interface, and ODK form. Both English and Bangla versions were available for participants. Image courtesy of Andrew Bell.

Making smartphone tools intuitive will play a big part in helping their diffusion, and in turn, mainstreaming the amenities many of us now take for granted (such as mobile banking, maps, or social networking tools). In particular, survey tools that are intuitive for people in rural areas to opt into can provide a powerful mechanism to channel research dollars more effectively (via rewards like airtime, data, or anything else that can be credited to a mobile device) into benefits for the communities whose lives we are hoping to improve through observation.

Recent posts