Mission 4636 and improving the Mechanical Turk

During the days after the earthquake in Haiti, I was traveling and was not able to participate with the ongoing mapping efforts at CrisisCamp.  However, I am reasonably fluent in Haitian Creole/Kreyol so I volunteered to translate.  I was very happy to discover that Ushahidi had set up a SMS translation service called Mission 4636.

Mission 4636 is the ongoing effort to collect and translate SMS messages from Haitians and route their requests to responders on the ground. Mission 4636 started at Ushahidi and moved to CrowdFlower with Samasource for long term support. At this time, Mission 4636 relies on volunteers to provide the translation and geolocation services.  Samasource provides the following graphic describing Mission 4636:

source: (Samasource: http://www.samasource.org/haiti/)

The system is essentially a mechanical turk with volunteers performing the translation and extracting coordinate information based on references to addresses or more general locations in the text of the message.  Volunteers at CrisisCamps quickly mapped Haiti in OpenStreet Map and and other contributers created the Haiti Crisis Map which is helpful for finding the coordinates of locations in latitude and longitude. 

While tools such as the Haiti Crisis Map are important, it is the volunteer translators that make the system work.  For example, I translated a text with the address of Ri 3, Fre Rigo Okay to Rue 3, Freres Rigaud, Les Cayes, which is the French label on the map.  Haitian Kreyol is more of a spoken language with a loose orthography and has a tendency towards being a phonetic language.  Having lived near Les Cayes, I knew that it was called Okay in Kreyol, which was helpful for knowing where to look in the first place.  Similarly, there were many requests from Delmas, an area in Port-au-Prince, that required local knowledge to find the precise location of the person sending the SMS.  Finding the coordinates of these locations to accurately direct responders on the ground depends heavily on local knowledge that can not be derived from the map.  

Ushahidi created a chatroom for volunteers which was instrumental for tapping into and sharing local knowledge.  Having a place to interactively discuss location and provide assistance for translating the SMS greatly improved the translations and the accuracy of the location coordinates.  Being able to work collaboratively with others made the information in the SMS actionable and facilitate the work being done in country.

The importance of collaboration is highlighted in a recent article in the New York Review of Books by the chess grand master Gary Kasparov.  In the article, he describes a free style chess tournament where players could compete in teams and use computers.  Chess playing computers essentially brute force solutions to win, but typically they cannot defeat a strong human player.  It was expected that a strong human player (professional) with a powerful computer would be the logical winner of the tournament.  The winner(s) were a pair  of amateur players that collaborated using three consumer grade computers.  Kasparov describes the outcome as surprising:

"The surprise came at the conclusion of the event. The winner was revealed to be not a grandmaster with a state-of-the-art PC but a pair of amateur American chess players using three computers at the same time. Their skill at manipulating and "coaching" their computers to look very deeply into positions effectively counteracted the superior chess understanding of their grandmaster opponents and the greater computational power of other participants. Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process."

My experience with Mission 4636 validates that a combination of humans of varying skill levels + machine mediated interaction and tools + better process produces better results than having volunteers simply performing translations without the benefit of a collaborative approach.  While a mechanical turk system can harness the efforts of volunteers distributed globally, the addition of an adhoc method of collaboration greatly increased the efficacy of the system.


Ushahidi continues to host the chatroom which is the touchpoint for 4636 volunteers.  I hope that Crowdflower and Samasource will eventually add a chatroom to supplant the one at Ushahidi and continue to foster the collaborative environment.  Volunteers are still needed at Mission 4636. If you are a Francophone or Kreyol speaker, or if you just want to help, you can register at Samasource.