Ushahidi: The crisis cartographer

The best open source projects are born out of a genuine need and Ushahidi is no exception.

The evening of December 30, 2007, saw the swearing-in of Kenya’s new President, Mwai Kibaki. The result of the hotly contested election was unexpected and within minutes of Kibaki’s victory being declared and not that of his main opposition, Raila Odinga, there were protests in the streets of Nairobi. 

The situation rapidly escalated and the violence began with the massacre of over 50 women and children. The fighting spread and grew along tribal lines until a power-sharing agreement was signed at the end of February 2008. While irregularities in the election were acknowledged as the primary trigger for the violence, The Office of the UN High Commissioner for Human Rights later concluded much of the crisis was fueled by “discrimination, poverty and disenfranchisement”.

• Find the right flavor of Linux for you with the best Linux distros
• Get your site up and running with the best Linux web hosting
• These are the best Linux apps

Ushahidi sprang up during this difficult period in Kenyan history as a website that attempted to empower Kenyans that didn’t have a voice. Created by concerned bloggers and software developers, either living in Kenya or former residents, It tracked and mapped the post-election violence. 

Ushahidi combined social activism and citizen journalism and meant that anyone with a phone or internet access could text or email in eyewitness reports. (Ushahidi means ‘testimony’ in Swahili and is also related to the word ‘shadidi’ for ‘witness’.) Reports that were verified by sources, such as NGOs, Kenyan journalists and international media, were added to Google Maps and timestamped to create a visual testimony of the country’s unfolding crisis. Each report triggered alerts back to the people on the ground, so they could stay informed, access relief and hopefully remain safe, and to signpost to those watching from around the world what was happening to Kenyans. More than 1,200 people died and over 300,000 people were displaced during the crisis. In terms of reporting, Ushahidi was found to be more accurate, by the Kennedy School of Government at Harvard University, at reporting the break out of violence than Kenya’s own mainstream media outlets.

While Ushahidi’s emergence was deemed a success both nationally and internationally, Will Doran, senior developer and Director of Technology at Ushahidi, says the ‘platform’ started out as little more than a WordPress blog “with dots on a map”. Over the years, the Ushahidi platform has evolved and been utilised in significant relief efforts: after both the 2010 Haiti and Chile earthquakes; for monitoring the 2011 Nigerian elections; unrest in the Balkans and, of course, further Kenyan elections. The Ushahidi organisation itself has become a respected social enterprise based out of Nairobi with over 30 staff scattered across the globe — Doran himself lives in Canada. 

The latest version of the platform, v3, is built on Laravel's PHP framework and is now able to collect data from multiple sources including email, mobile apps, SMS and Twitter. The data management tools have advanced as well with filtering and saved search features, and teams are able to create workflows and collaborate together more effectively to manage and triage reports as they come in for much quicker report verifications during a crisis.

Ushahidi’s data visualisation is also much more than pins on a Google Map now. It supports multiple map tile sources including Open Street Maps and sucks in data streams from a variety of third parties and presents a lot more information using configurable charts. An alert system has also been developed to flag changes to a deployment for admins and for general post updates.

Open source roots 

Today, as it celebrates over 11 years, Ushahidi has grown into an “entire ecosystem of software and tools” Doran told us, on a VoIP call from Canada, and receives grants and funding for numerous human rights projects, such as COMRADES, a European Commission funded project to build a platform to support community resilience during a crisis. Everything they do now, Doran says is “built to facilitate the work done by human rights advocates, journalists, election monitors and those responding to disaster and crisis.”

Doran is a recent recruit having joined Ushahidi three years ago. He’d previously been working on making open source tools for censorship and surveillance circumvention, but with a background in machine learning Ushahidi had a particular draw: “I was interested in applying some ML to more kinds of data that people are able to collect themselves. One of the things that's fascinating about Ushahidi to me is that a lot of tools try to lower the technical barrier to entry. Ushahidi was managing to do that, so huge numbers of people were spinning up their own instances of v2, or Crowdmap Classic as it was called externally.”  

A large number of groups from all areas have used and continue to use the platform from crisis response, emergency rights violation documentation and even people interested in the price of chickens: “It seemed to fit a really good niche usecase that was a bit more feature-involved than a blog and gave people slightly more ability to do some data analysis in the form of reporting,” says Doran. “Traditionally, in the human rights space, you need a larger established org to come and write a report about the situation that you're experiencing in order to draw international attention.That's still the case to a certain degree, but what was interesting to me about the Ushahidi platform was people's ability to collect their own experiences and to begin to publish and support that with data, so you could start to make your own compelling arguments about experiences you and others around you were having and how your lives were impacted.”

Much of what Ushahidi has achieved has been with the help of open source software and being open source is a fundamental tenet for the enterprise (v2 of the platform was released under the Lesser GPL while v3 uses AGPL).   

Doran says Ushahidi’s tools are built from and use a mix of Python, PHP, JavaScript, Ionic, AWS, Codeship, Travis CI and Github repositories. Although Ushahidi has changed its model recently by offering deployments as a paid service, the platform is available in its Github repo for anyone who wants to self-host and ultimately Doran would like to make cloud deployments a possibility in the future as well: “Our deployment systems are built on Ansible and Terraform now and we’re working towards making that fully open source so that people can generate their own cloud instances if they want,” says Doran.  

Ushahidi uses a tiered model if you can’t self-host where any grassroots organisation can apply for free use: “Essentially $250,000 is the cut-off, Doran explained. “So if your organisation receives more funding than that then we would ideally like people to pay.”.

As it stands, making Ushahidi sustainable is a question of holding many things in tension: “Ushahidi is still primarily grant funded and probably likely to be for the foreseeable future. In terms of sustainability, in terms of selling the software, it is difficult because you're, on the one hand, competing with your own free software and you're trying to find a level where you are mostly providing the support and training services. That's what I think most people are interested in is the expertise around the software and then also, ideally, a good quality software product that people want. It's about finding that balance.” For now, the main areas that produce revenue for Ushahidi are client work and customisation for larger NGOs or organisations that are implementing a project.

Evolving over time

Over the years, the platform has undergone a number of framework changes. The current stack is built from the Laravel PHP framework and AngularJS. It used to be Kohana, the PHP5 HMVC framework, but that, as Doran notes, is "deceased now". While Doran agrees that cost-wise open source makes a massive difference, it's more than that for them: “It gives you more control over what's happening [...], in particular, in terms of privacy and security.” and while Ushahidi makes quite a lot of contributions to Laravel, it’s committed to open source “generally everything that we build that we can open source we do.”

“I think software like this needs to be generally freely available to people because that facilitates the development of interesting organisations that would otherwise be hamstrung.” Most grassroots groups don't start out with funding or grants and even if they do, it's usually for specific costs, such as travel: “They may have funding for their research or to pay themselves so they can actually do the work,” says Doran. “so our interest is in making the software always available to those groups.”

A look at Ushahidi’s Github repo highlights just how extensive the list of products and platforms is that have grown out of the initial idea. The Ushahidi platform still sits at the centre, but around it is TenFour (a team check-in tool for confirming the safety of colleagues during a crisis) and SMSsync for turning an Android phone into a local SMS gateway for sending incoming SMS to a configured URL. It’s even led to spin-off companies such as BRCK, which designs solar-powered tablets and routers which have a focus on connecting frontier markets, a market which is expected to connect three billion users to the internet.

False witness

Any venture that relies on crowdsourcing information is susceptible to exploitation and fake news, which is why Linda Kamau, lead developer at Ushahidi, says the verification part has been more a human than a technical process in the past. Kamau’s passion is monitoring Kenyan elections: “we don’t automatically go marking reports. There will be a team that does verification, so, for instance, during our last election in Kenya, we worked with an organisation that does election monitoring.” This was a partnership with InfoNET and The Constitution & Reform Education Consortium (CRECO), where they set up a deployment called Uchaguzi (‘election’ in Swahili). CRECO, says Kamau, would check with their monitors on the ground to verify reports of electoral offences and peaceful as well as violent activities. 

To automate some of the verification processes, Ushahidi has been running a series of projects funded by the European Commission. PHEME, for instance, is a rumour tool which performs sentiment analysis on text data, specifically in tweets but generally in text. “Its work was attempting to look for rumours and then either reputation or confirmation of rumours or confirmation of facts,” says Doran. Another project, COMRADES is more focused on crisis response categorisation of inbound data. COMRADES is a platform designed to help communities reconnect, respond to and recover from a crisis situation.  

The direction Doran hopes to go with the COMRADES work next is addressing falsity; quantifying the extent of the falseness of statements, then onto actionability and validation, but he’s quick to point out that the machine learning tools would never supplant humans: “One of the key things we're trying to solve is how do we communicate to people what the machine thinks is its error rate or its accuracy is so that they can then make decisions.”

As it stands, Doran can’t confirm to what extent the platform has seen significant quantities of 'fake news': “It's difficult because traditionally when Ushahidi does actual monitoring it's for a very specific period, a time window around the election [...] I think in the future that window needs to expand, because of a lot of this information happens over a much longer range - at least six to eight months beforehand. Purportedly, the English company Cambridge Analytica was consulted at least by one of the parties during the Kenyan election, but detecting that is a whole different level.”

Help break Ushahidi

In order to move from v2 to v3 of the platform, Doran acknowledges that they were focused on internal development, “but we want to move back to a space where it's easier for people to contribute features that make it all the way to the app.” Generally, Ushahidi is interested if people want to contribute code. “We're defining a pathway for that, but the code is relatively well structured. If people are interested in reading through a feature list or a bug appeals to them, usually if they send in a pull request we'll review it and probably integrate it.”

Testing is one particular area Ushahidi needs help in: “The more people that can test, ideally in multiple languages, and in different circumstances and break it and tell us how they broke it - that's hugely valuable to any open source project,” says Doran

Ushahidi’s open source documentation is also being beefed up. A surprising result from 2018’s survey highlighted that over half of the open source self-hosters were struggling to deploy v3 of the platform. In response, Doran says, it's running a project with DIAL Open Source Centre, which is funding Ushahidi to help develop our open source documentation with the intention of creating a better pathway for people into the platform.

Ushahidi may have celebrated 10 years last year, but Doran sees a lot of work ahead and is keenly aware that there’s still quite a technical gap: ”People are not homogeneous and different groups have different working methodologies, and I think the software better supporting and facilitating their work, so working for them is what's really important. For me, that's the next 10 years of Ushahidi; It's that the users have more and more of a sense of ownership of the tool. And I think that's what most open source projects really want.”

The line is crackling from Canada and we’ve been talking for over an hour, but as we finish up Doran says he tries to attend the Internet Freedom Festival in Valencia each year. It’s an opportunity for him to meet a broad mix of people in the NGO tech field and people working on the ground and activist organisations. Part of the event involves a high-profile tool showcase, where it’s all about “what cool tools can we build to help people’s lives,” but he says it’s transitioned in recent years for the better to ask questions with a different focus like “how do we use these tools for them to be a benefit? Are they actually a benefit or how do we make better tools that are driven by people’s actual needs?” Ushahidi was born out of need. That need was to give a voice and aid to the marginalised in times of crisis and it will continue to do that until it runs out of ways to fund itself.

In January 2019, Ushahidi received funding from the Rockefeller Foundation to keep evolving and scaling Ushahidi into a “collaborative reporting and automatic response platform for NGOs, governments, and communities around the world”.

• New to Linux? Check out the best Linux distro for beginners