Kenyan Election Open Data

Election boundary and polling station geo data for Kenya can now be downloaded.

Kenyans are gearing up for the presidential election in March, including the tech community projects like Uchaguzi and Map Kibera. One limiting factor is availability of data. The Independent Electoral and Boundaries Commission (IEBC) finalized new constituency boundaries earlier this year, but have only released non-machine readable pdf maps. That’s led Map Kibera to resurvey on the ground the new boundaries and polling places, in Kibera, Mathare and Mukuru. But outside that excellent work, open data on the most primary geographies of democracy are not available for Kenya. Election boundaries should be the number one data available on OpenDataKE.

The IEBC released a nice site to look up Kenyan polling places. It doesn’t directly offer download, but quickly looking at network requests for the app revealed simple endpoints to request json versions of constituency and county boundaries, and the locations of polling places. I wrote a script which iterates through every ward in Kenya, caches the data, and produces outputs. There are shapefiles for download here. If you need something tweaked, let me know, and I’ll see what I can do.

This map displays approximately 16000 polling places across Kenya. In total, there are 26447, but a significant number are not mapped. Also, there are no ward boundaries available. This is a big opportunity for Kenyans to contribute back to their government’s data. For instance, Lindi Mosque has no location in the government data set, but is present in OpenStreetMap. Would be possible to repurpose some OpenStreetMap microtasking tools to quickly map these places (HOT Tasking Server, MapMill or MapRoulette).

Why make this data open? So we can build things like
* Send an SMS to get a list of polling places in your ward
* Geocoder and custom base maps to make it easier to locate reports to Uchaguzi
* Print map services, so you can easily distribute polling place locations in your area
* Analysis, such as distribution of polling places according to population density
and plenty of creative things I haven’t imagined yet.

I hope this step of action, then begging forgiveness rather than asking permission (in I hope the best tradition of mySociety) will spark curiosity and consideration by the IEBC to more fully embrace openness, not only by releasing data, but inviting conversation and feedback from the citizens it serves, and becoming more transparent in how decisions are made and how the complicated and incredibly important business of running an election is being carried out.

Moabi and Big Important Challenges for the GeoWeb

Recently finished a technical review of WWF’s Moabi platform, their geodata sharing platform to track deforestation, and came out with some important technical challenges for the GeoWeb I want to share with the mapping nerds.

Just following TAI’s bridging session on natural resource governance, I met Charles Huang of the World Wildlife Fund at a similarly styled event at the World Bank. After self-identifying as a developer, Charles asked if I’d like to talk more about Moabi … they needed a developer’s eye on the platform as they considered how to move to their 2.0. This isn’t GroundTruth’s usual sort of work, but considering the TAI Bridge effort to connect technologists with sector experts, our other work with Drupal and mapping, and my own deep interest in conservation, I dove in.

Moabi is ambitious. In its current expression with DRC and deforestation, the platform is meant to collect geographic data and detailed profiles from all sorts of courses, including the “crowd”, on everything from mining concessions, informal logging, road construction, to REDD projects, and enable conversation and coordination among everyone conservation workers on the ground, government officials, and the interested public on the other side of the world. And in the long run, this platform is intended to apply to other places and focus areas of WWF. This is such an ambitious, wide mandate, and I think touches on 3 key technical developments for the GeoWeb.

Features as Full Social Objects

In Moabi, every single feature, such as an individual mine or road, has a profile page, which should be shareable, pivotable, a focus point for social interaction among people and groups organizing to prevent deforestation. Most often in GIS and web mapping, we talk about layers of information, and while individual features often have expression in various forms on a site, there is no open source software package that deals with feature-level data in this way out of the box. And there’s not really that many examples to draw on. Every feature in OpenStreetMap has an individual url, with details on its tags, last editor, etc. But it’s not quite used as a social object, excepting maybe the awesome machine tag integration with flickr. Foursquare has some elements of this, every venue primarily being a place, and users able to connect to that place. But besides these sorta examples, a geographic feature is generally treated as a property of social objects, rather than social objects in themselves.

Multi-master sync and flexible schemas

Data is Moabi is sourced from both government, civil society, and interactions on the site. Those source databases are very likely to update over time, and very possibly they will receive updates on Moabi as well. Given a foreign key, or even a clever lookup, at least identifying which features have updated is possible. But this inevitably leads to thorny issues of handling conflicts during synchronization. This is commonly experienced in the simple situation of editing in OSM during a mapping party, where two editors independently modify the same feature. The OSM API can detect the potential conflict, but once detected the interface and process to resolve the conflict in JOSM is not intuitive or easy at all. When writing code, this situation happens all the time, and simply being text, it’s simple enough to throw the problem to the programmer to resolve. But we’re talking about geographic features here, and that’s going to take very well designed interfaces to solve properly.

A related issue is that a single feature may contain pieces of information from several different masters. Take roads. Yes, common enough is the geometry, maybe the name (or names) of the road, the surface. But with Moabi, they may also want to know if the road has a regular anti-poaching patrol, or if it’s commonly used to transport lumber. OSM deals with this unpredictability of attributes through tags, arbitrary key-value pairs, with no predetermined mandated structure, with conventions of use worked out over time. In contrast, traditional GIS requires pre-decided database schema, leading to long running discussions which often never resolve to the point of actually collecting and sharing data. So why not just use OSM? Well I’m not sure it’s best to keep every single thing interesting to Moabi about roads in OSM, and some features, like say mining concessions or REDD projects, may simply not be appropriate for OSM (this is up for discussion of course). So assuming that something else is needed, what are the options for a tagging based system that’s not OSM? The closest I’ve seen is Matt Berg’s work at Earth Institute on georegistery, an open source db with RESTful API for sharing and versioning geographic objects in GeoJSON, with arbitrary attributes.

Complex interactive experiences without a lot of fuss

Moabi has a lot of data, two dozen layers, thousands of features. Currently the only way to work through the data is via query, which can be a little slow both in usability and response time. Interactive exploration is best accomplished through Tiles and Grids, as enabled so well in TileMill and MapBox. And there’s been a lot of activity in OpenLayers with Grids, including support for interaction in multiple layers.

The issue comes with how to make this architecture efficiently queryable. Say you wanted to show only to show mines operated by Chinese companies in the DRC, which are not located in mining concessions. The most obvious way is to send this query to the server, and render the result on the client side. But perhaps there are other ways, utilising the same approach MapBox Streets takes to customize styles on the fly, by manipulating the color table of tile images. If a tile set handled just a single layer, then the color table could someohow be used to encode other properties. This would essentially pre-can a set of interesting queries during tile generation, with the query results rendered by the server. If that’s too limiting, perhaps we need to start thinking further in the future, where something like SVG based tiles could be efficiently rendered and queried in the browser.

Thoughts?

Would love to hear feedback on whether these are actually interesting challenges in your opinion, and what are some other ways of moving forward with them.

When it comes to politics, crowds can be manipulated, but not communities

The women and children in the photo are suffering, and the story tells of hidden revolutionaries challenging brutal rule in all arenas, and victorious on the maps of the most powerful company on the planet. They’re renaming the infrastructure after revolutionary heroes. You can’t help but cheer on such clever efforts for freedom.

Google supports liberation! Another front in American online diplomacy?! How far from the truth. Another lame attempt to boost American companies sales with puff pieces about their support for the Arab Spring. Let’s watch and see how long before Google scrambles to show its commitments to national governments (i.e. customers) and their maps switch back to their Assad era names.

The real story here is the nature of “crowdsourcing” (a term I’m increasingly despising), and power over and control of our geographic reality. Stefen Geens says that such false information, that politically motivated editing is a risk of crowdsourcing; it’s not, it’s rather the result of a false community and opaque processes. This write-up (“regime change, hardly”) is an excellent blow by blow, but there is absolutely no way to full penetrate this proprietary system.

Could this have happened on OpenStreetMap? Sort of. Anyone can edit anytime, I could change these names right now. The difference is that the change would be spotted soon by the community which cares for this data, all past changes by the user easily identifiable, discussion and questions posed in public, and reverts applied if necessary. In the event that the inaccurate edits ontinue, the case can be escalated to mediation, and the DWG can finally take actions like warning and blocks. It’s happened before, in Northern Cyprus, and OSM dealt well.

OpenStreetMap does not support the Assad regime, nor does it support the rebellion. It supports everyone’s access to the facts, and the equal ability for those common facts to respond to reality. OSM was the first map to display the world’s newest country of South Sudan. And in the event the brutality in Syria ends, and the streets are renamed on the ground, you know where to edit.

Only Possible With Open Data

Arguments about the importance of Open Data often come down to a principled stance, or a licensing discussion … that kind of argument doesn’t make much impression on folks who aren’t way in the weeds. And it’s more than just licensing … there are equal parts issues of legality, technical freedom, and community. Clear examples of what you can do only with OpenStreetMap, and not with say, Google Map Maker, makes this stuff real. Here are just a few, among many.

Mapping of Jalabad and surrounding countryside is unique to OSM. They collect data with GPS and Smart Phones and Walking Papers. With GMM, you can only trace imagery on your laptop computer. With OSM you can go into the field to actually talk with people about what to put on the map. Afghanistan is not an option at all on MapMaker, for political reasons. And the Jalalagood guys are organized as a company, but do mapping largely in their free time, voluntarily … so they get nixed for being “commercial”.

Great video on Jalalagood and a detailed mapping trip report.

In Haiti, TapTapMap maps local bus routes, free to add any sort of data in OSM. In GMM, users are not allowed to add bus routes, only Google does that, and only by getting data from transit agencies. Haiti’s system, like many places in the developing world, does not have a single authority which maps and controls the routes. You can only get these routes by riding with your GPS.

Another project that certainly couldn’t happen without Open Data. This is a Tourist Map of the Gaza Strip. Produced by a spin-off company from a university that took part in OpenStreetMap mapping in Gaza. This is not an area that can even be mapped in Google because of a political decision, it’s mostly blank on the map. While the map is free, it’s produced by a commercial entity and contains ads for local restaurants, hotels and sites.

In the Phillippines, recent humanitarian response to flooding relied on open source tools to process and make available satellite imagery to create OSM data. The GMM toolset would not permit integration of any other data sources except those controlled by Google. So it would stifle local ability to respond to disasters that don’t make huge media splashes (G Crisis Response has not been active at all in the Philippines).

A Week for the Record Books

That was a week where I really was truly and completely welcomed to DC. Some great things, some other things, and some things I can’t talk about yet.

Was invited to fill for Kate Chapman at the Mapping the World of Humanitarianism workshop (not the geo kind of mapping). Talked about the challenges with community centered social technologies, within the humanitarian system. Most difficult question “Do you consider yourself a humanitarian?”

Seda Muradyan invited me to present within a multi-day workshop with Armenia journalists. Was via Skype, with translation (the translator did an amazing job. Spoke about OpenStreetMap and how we’ve applied it within community-centered journalism projects. Just missed saying hi to Noha Atef who had been in Armenia for this. Still amazed that the Internet makes this kind of thing possible.

TC103, Tech Tools for Emergency Management is an online course led by TechChange. I’ve been moderating, and the quality of the discussion and participants is phenomenal. Rob Baker and I sat down for a chat in front of the camera, good fun … may be released at the end of the course.

World Bank and Google

* http://www.nytimes.com/2012/01/14/opinion/empowering-citizen-cartographers.html?_r=2
* http://www.globalintegrity.org/blog/bank-responds-to-google-maps-deal
* http://irevolution.net/2012/01/20/google-inc-world-bank-empowering-citizen-cartographers/#comment-11284

How many beers, coffees and phone calls discussing this boggling move last week? I’m still thinking about how to respond and where to go from here.

OpenStreetMap and Google

Google contractors were caught vandalizing OpenStreetMap. Oy vey.