Kenyan Election Open Data

Election boundary and polling station geo data for Kenya can now be downloaded.

Kenyans are gearing up for the presidential election in March, including the tech community projects like Uchaguzi and Map Kibera. One limiting factor is availability of data. The Independent Electoral and Boundaries Commission (IEBC) finalized new constituency boundaries earlier this year, but have only released non-machine readable pdf maps. That’s led Map Kibera to resurvey on the ground the new boundaries and polling places, in Kibera, Mathare and Mukuru. But outside that excellent work, open data on the most primary geographies of democracy are not available for Kenya. Election boundaries should be the number one data available on OpenDataKE.

The IEBC released a nice site to look up Kenyan polling places. It doesn’t directly offer download, but quickly looking at network requests for the app revealed simple endpoints to request json versions of constituency and county boundaries, and the locations of polling places. I wrote a script which iterates through every ward in Kenya, caches the data, and produces outputs. There are shapefiles for download here. If you need something tweaked, let me know, and I’ll see what I can do.

This map displays approximately 16000 polling places across Kenya. In total, there are 26447, but a significant number are not mapped. Also, there are no ward boundaries available. This is a big opportunity for Kenyans to contribute back to their government’s data. For instance, Lindi Mosque has no location in the government data set, but is present in OpenStreetMap. Would be possible to repurpose some OpenStreetMap microtasking tools to quickly map these places (HOT Tasking Server, MapMill or MapRoulette).

Why make this data open? So we can build things like
* Send an SMS to get a list of polling places in your ward
* Geocoder and custom base maps to make it easier to locate reports to Uchaguzi
* Print map services, so you can easily distribute polling place locations in your area
* Analysis, such as distribution of polling places according to population density
and plenty of creative things I haven’t imagined yet.

I hope this step of action, then begging forgiveness rather than asking permission (in I hope the best tradition of mySociety) will spark curiosity and consideration by the IEBC to more fully embrace openness, not only by releasing data, but inviting conversation and feedback from the citizens it serves, and becoming more transparent in how decisions are made and how the complicated and incredibly important business of running an election is being carried out.

Only Possible With Open Data

Arguments about the importance of Open Data often come down to a principled stance, or a licensing discussion … that kind of argument doesn’t make much impression on folks who aren’t way in the weeds. And it’s more than just licensing … there are equal parts issues of legality, technical freedom, and community. Clear examples of what you can do only with OpenStreetMap, and not with say, Google Map Maker, makes this stuff real. Here are just a few, among many.

Mapping of Jalabad and surrounding countryside is unique to OSM. They collect data with GPS and Smart Phones and Walking Papers. With GMM, you can only trace imagery on your laptop computer. With OSM you can go into the field to actually talk with people about what to put on the map. Afghanistan is not an option at all on MapMaker, for political reasons. And the Jalalagood guys are organized as a company, but do mapping largely in their free time, voluntarily … so they get nixed for being “commercial”.

Great video on Jalalagood and a detailed mapping trip report.

In Haiti, TapTapMap maps local bus routes, free to add any sort of data in OSM. In GMM, users are not allowed to add bus routes, only Google does that, and only by getting data from transit agencies. Haiti’s system, like many places in the developing world, does not have a single authority which maps and controls the routes. You can only get these routes by riding with your GPS.

Another project that certainly couldn’t happen without Open Data. This is a Tourist Map of the Gaza Strip. Produced by a spin-off company from a university that took part in OpenStreetMap mapping in Gaza. This is not an area that can even be mapped in Google because of a political decision, it’s mostly blank on the map. While the map is free, it’s produced by a commercial entity and contains ads for local restaurants, hotels and sites.

In the Phillippines, recent humanitarian response to flooding relied on open source tools to process and make available satellite imagery to create OSM data. The GMM toolset would not permit integration of any other data sources except those controlled by Google. So it would stifle local ability to respond to disasters that don’t make huge media splashes (G Crisis Response has not been active at all in the Philippines).

Good Citizenship in Open Data

I didn’t expect my rant would strike such a nerve, but maybe I’ve been holding my tongue for a long time. Thank you for the many responses, online and off, both supportive and challenging.

We must work to understand what good citizenship and ethical behavior means in open data projects. The nature of communication, copying and competition in the space of open data is very complex. Yes, it’s not just about Google, but about raising the awareness of these issues among the people organizing open data projects, and especially the communities where we want to have an impact. The best idea I’ve heard this week (in a week of amazing ideas in Cambridge) was from Jeffrey Warren. We need a clear set of principles and ethics to guide the practice of open data initiatives in new communities. Open data collection should have: open and clear explanations of the purpose of data collection and the license of data; effort to find existing sources of data, rather than replicating and resurveying, and lobbying for the sharing of that data; effort to give the communities that collect data every opportunity to use that data in their own work, however they see fit; etc…

Participation in open data is both about the arcane topics of data licensing, and simply and most importantly being a good citizen. The impression of legal matters is usually all any non-lawyer grasps … it’s just too hard to understand the subtleties of legal text, and we rely on trusted intermediaries to explain what’s what. Google is trusted, and they’re misusing that trust with communities, NGOs, and governments. And Google has huge resources, they get a lot of attention. They’re representing their mapping offerings as free and open, when they’re ultimately controlled by their business concerns. Google has one of the slickest communication practices around, with subtle phrasing that gives an impression of cooperation, but with built in protection whenever someone digs into the details. This is deceptive.

There’s no problem with copying, it’s encouraged. Mapping Parties are not patented. There’s no license on ideas, because fair play is expected. Simply, being a good citizen, being a friend in this diverse space, means giving credit and giving back. Google has taken a concept, adapted it, and not attributed. Not until last year did they even use the phrase Mapping Party for Map Maker events (sometimes that got the attention it deserved). In places where there’s been exposure, Google didn’t “throw Mapping Parties”, because of course it would’ve been noticed as deriving from OSM. And Google is alergic to giving any notice to OSM, for example omitting any mention of the biggest open data project on the planet, which originated in the UK, from their glossy magazine on UK Open Data.

Competition is a very crude and inaccurate way of describing the relationships in the open data space, especially between OSM and Google, and is based on confusion of what OpenStreetMap is all about. OSM is essentially a commons, a data project where everyone can contribute, share and use. OSM is not a company, it’s not an NGO. It’s a collective where hackers, governments, universities, anarchists, slum dwellers, multinational corporations, and international organizations all contribute to a common endeavor to map the world. Every other major internet company contributes to OSM in some substantial way. It is Google only that has decided it’s in competition with OSM, not because of the designs of engineers and product people, but because doing so would undermine something Google thinks it can profit from.

Honestly, I would be thrilled to see Google adopt the use of OSM. The data of OSM is very different from the interfaces and the software. The site and tools definitely need improvement, but it’s not because wants to take away traffic from Google Maps. OSM’s frontpage needs to do something different: “The frontpage fails to communicate what the project does and why it does it effectively. The controls are confusing for new users (lots of random tabs and sidebar). In addition, it totally fails to connect new users to others around them and thus create a community.” Google could even be helping with this.

Never did I say that Google should stop making its products, but rather than it should stop exploitation. In more positive terms, Google must be a better citizen. I even said that a lot of what Google is doing in Africa is good for Africa. I use Google products all the time, live by Google Calendar, and think Summer of Code is awesome no matter how many spaces OpenStreetMap gets. They’ve contributed in great ways to humanitarian response. I have many friends at Google, who are not surprised by what I said, but perhaps surprised that I said it out loud on the Internet. But this is one place where Google’s Africa Strategy has fouled up directly in a space I care deeply about, fueled by business concerns, cloaked in open data clothes, but ultimately dangerous. As Ping said on Crisis Mappers (about a different OSM-Google issue, but equally valid here) …

It is okay to have issues with Google. Google is a big place; there are a lot of different people and different teams here, all trying to accomplish different things. It’s tough to assign a coherent intent to the entire company; we’re a bunch of people trying to work together even though we don’t always agree. No individual person is trying to be evil. Sometimes we screw up, though.

So yea, I’m pissed off, and can’t keep quiet. And actually, as some have asked, there is a lot more to these stories that you’ll only here about over a beer. If you only know me by this post, perhaps you’re surprised to hear that I’m usually known as a mediator, diplomatically open to all sides of an issue, generally a nice guy who just wants to map the world. And in case it wasn’t obviously clear, this all is my personal opinion. Another aspect of our open world is our multiple public representations, and I wasn’t speaking for the OSM Foundation, Humanitarian OSM Team, or GroundTruth. And yes, the OpenStreetMap community has a reputation for being “caustic”, which is based on a very small segment of our community, mostly the osm-talk list. Anyone who has ever attended State of the Map, gone to a pub meet, contributed to a HOT response, posted to nearly any other list, will have a different impression.

Finally, I regret to say that Unosat got caught in the cross-fire of this post. Unosat has an evolving cooperation with many entities, including HOT, where we have been given access to imagery in Japan and Liberia. While I find fault with their Google MapMaker cooperation, they aren’t giving Google any data, rather using Google data. They are well versed in the details of open data, of licensing, and are wise to explore opportunities equally. They’ve generously explored how they can work together with OSM, most recently in Japan and I look forward to more.

We Need to Stop Google’s Exploitation of Open Communities

Google’s strategy is to build market in Africa by appropriating the appearance of open data community methodologies, yet maintaining corporate control of what should rightfully be a common resource. They are specifically targetting govts and NGOs, offering to “map their country for free”, but keeping the results, and attracting customers.

What bothers me so much is how they have blatantly copied OpenStreetMap. First their MapMaker product is directly modelled on OSM, but with a restrictive data license, where you can not use the data as you see fit. Second, they have stolen the idea of Mapping Parties, a unique concept and name we developed. Third, they’re even copying initiatives to map impoverished informal settlements, like Map Kibera.

All of this with no credit, and no shame. I’m sick of Google. What can we do?

I’ve tried to stay civil with them, through years of this kinda stuff. I’m friends with many googly Googlers. But I don’t see cooperating or being quiet helping at all. They’re deceptive and need to be stopped.

Seriously, Google claimed to map “the largest slum in Africa”, with “citizen cartographers”. They’re building their business by glorifying half-baked “community” mapping initiatives, promoting their brand on the back of poverty.

I totally get why African governments and techies are excited about Google. Their moves in Africa are going to help build up the market for everyone. What is horrible is the hidden dangerous bargain they’re offering. To most people, Google is not just a company, but a force for good in the world. They even forget its a business, with so much done for “free”. But remember, it’s an extremely lucrative business, and they reason they don’t participate in OpenStreetMap, like every other major internet company (AOL, Microsoft, Yahoo…), is because it doesn’t make sense for their bottom line. They see value in owning your data. They’re moving to own the data of communities and governments in Rwanda, Kenya, Zambia. They can do whatever they like with the data, they own it. Of course, they will eventually take advantage of that. Scratch eventually, they changed the terms on Friday, so you have to pay to opt out of GMaps API advertising.

Corporations should not be the stewards of a public resource, and a potentially controversial public resource. Compare Gaza in OpenStreetMap and Gaza in Google for just one example of why this is a bad idea. We’re approaching a situation where a corporation is becoming the decision maker on international borders. Wait, did you think the UN or other international forum was supposed to have some role in these kind of things? Nope, Google is getting UN data too.

Internet, now’s the time to get pissed.

First Thoughts on Google MapMaker Download

Google announced today MapMaker Data Download. What does it mean for OpenStreetMap and Open Geo Data?

The early chatter is focusing on the licensing, naturally. Non-commercial use only, non-compete with Google services, and strict attribution requirements. All three mean MapMaker data is incompatible with OSM licensing (cc-by-sa, or the ODbL). The meaning of non-commercial is pretty clear. Non-competition clause with Google means things like OpenCycleMap are not possible, as routing is explicitly not permitted. But does it mean you can not produce tiles and use them in your slippy map? What about in a mobile device?

What’s left? Well, some humanitarian response applications are possible. UN-OCHA and UNOSAT could make maps with MapMaker data .. but they could not make routing apps for relief workers. Many academic projects would be clear. There’s many possible uses for project in the public interest. It is encouraging that Google is moving in this direction and willing to help some class of mapping projects.

OpenStreetMap is definitely not satisfied .. MapMaker is still effectively a data silo. It is more useful than before for sure. And that’s a distraction from the goal of truely free and open geographic data. Google’s omni-presence and still huge budgets and comprehensive satellite imagery mean they can muscle their way into the discussion. It’s typical Google satisficion.

The data is very interesting. Quickly comparing the roads layers against OSM in Kenya show good correspondence where there is Yahoo aerial imagery to trace against — if OSM had access to the same imagery, I imagine we’d be pretty much equivalent country-wide. MapMaker is slightly more complete in central Nairobi; I put that down to Google having full time employees in Nairobi who work on MapMaker.

Some screenshots from overlaying OSM (green) and MapMaker (purple) in QGIS (you tell me whether this violates any licenses). This is just roads, plenty more analysis to be had on this and other layers.

Kenya MapMaker vs OpenStreetMap


Kenya MapMaker vs OpenStreetMap

Greater Nairobi

Kenya MapMaker vs OpenStreetMap-3

Central Nairobi

The MapMaker metadata file, included in the download, is fascinating and funny. The geographic features layers schema includes a TYPE_SHRUBBERY, POIs include TYPE_GURUDWARA (amenity=place_of_worship,religion=sikh in OSM).

We’re in the early hours of semi-open MapMaker. What will the future hold. WhereCamp Africa should be that much more interesting this weekend .. if you have any questions, leave in the comments and I’ll forward on to attendees in Nairobi. Will MapMaker data for other countries be released? Is tiling ok? What’s going to be the first MapMaker app? Will they import public domain data, such as the upcoming release of JumpStart’s map of Palestine?

Btw, there’s an upcoming Google MapMaker User Conference. Anyone from Bangalore planning to go?!

Wrapping up UNGIWG

UNGIWG concluded with another half day of openness. In the morning we heard about the geo activities of our hosts in Vienna .. UN Office of Outer Space Affairs, Comprehensive Nuclear Test Ban Treaty Organization, International Atomic Energy Agency, UN Office on Drugs and Crime. The CTBTO presentation was especially fascinating. They conduct one of the most important jobs at the UN, and for monitoring nuclear tests have deployed and leveraged several global sensor networks. The seismic fingerprint of nuclear tests are distinct from earthquakes. Noble gas isotopes are clear signs of a test. Infrasound and hydroacoustic measurements are also deployed; a nuclear test should register on all the sensor networks. The details on locations and functioning of the sensor networks are clearly communicated to the public in this elegant map. The information infrastructure to receive and process this flood of data streams is formidable, with 5 dedicated satellites and extremely minimal distribution lag to all the signatory countries, and analysis and permanent archiving in the Vienna center. The requirements are stringent , the solution robust and well funded. Of interest to the wider sphere, the archives are made accessible for unrelated activities, like tsunami early warning, volcanic blast detection, and climate change monitoring, all a wise example of reuse and repurpsoing of data. It’s not completely *open* but demonstrates how data access results in unexpected uses.

Lightning talks filled the rest of the monring. I gave two, and almost three. An introduction to GeoCommons led nicely from the previous days discussion of neogeo vs sdi. I gave a brief overview of the system and philosophy, with a brief example derived from Sean’s demos for the Gates Foundation. Good to see the acknowledgment that GC is bringing the accessibility of web tools with the analysis and cartographic care of traditional GIS. I took slight advantage of my 2nd lightning talk on OSM, adequately covered the day before, to switch and give a brief demo of Crabgrass and the work I’m doing at the UNDP. UNGIWG meets infrequently in formal meetings, and there’s scope for more sustained, agile, deeper contacts. There are an estimated 500-1000 GIS professionals in the UN system, and most are unaware of the work of UNGIWG, with extremely demanding work schedules and little opportunity to draw lessons from the wider community. My feeling is that a social network infrastructure is ideal to facilitate SDI, the human context is crucial when developing machine readable interfaces, and especially if that platform is open source and being deployed already in UN agencies. We’re currently working to set up an UNGIWG crabgrass, one of the brightest outcomes of the event for me.

Open source still faces difficulties in the UN. It’s commonly acknowledged that open source solutions, especially for geodata and maps on the web, are superior than proprietary offerings, but the commonly heard refrain is that the IT management is hostile to deploying open source. The expectations and accounting for software is biased against OS solutions, a grave irony when OS supports the objectives of the UN itself. Open source can provide valuable resources to beneficiaries just as a byproduct of UN requirements, and gives a technical basis for cooperation between UN agencies .. but this doesn’t happen that often. The must change, by appeal to upper management, something I hope is addressed at web4dev in February. There are bright spots of course. CampToCamp have developed camp management web platforms for UNHCR with open source tools, which I hope will provide opportunities for cooperation with efforts like Sahana (which has gotten some support from the UNDP), Ushahidi, and other crisis mappers, and the emerging OSM Corp concept.

Outside of UNGIWG, Vienna was lovely as ever and treated us well. The overflowing Baroque of Vienna is uplifting, the old masters and vaults of science, churches bubbling with iconography, even to the modern modern museums quartier, baroque is the guiding fabric (and home to a great cafe). The urbanity of Vienna is inspiring, imperial layered landscapes, imaginative restorations of the gasometer, enduring socialist idealism of Karl Marx Hof, speeding BMWs down cobbled luxury streets, walkable, cyclable, the best transport system anywhere. If only the city stayed open later. Even the brutal, only good on paper Vienna International Centre carries 60s UN hopefulness, and a reminder to remember the human scale even when buildings do not. That all was the scene for good times and some misadventure with old compagni and new of our geo vanguard.

And with a final sweetness, just dumb luck gave me the opportunity to facilitate a rather large geodata domation to OSM, more on that as things finalize.

UNGIWG, meat on the bones. OpenStreetMap and the UNSDI.

There is very much an emerging just $*!(% do it attitude. SDI East Africa has gone ahead within a specific geography and user base, started setting up the infrastructure of GeoNetwork and GeoServer and just put it into action; there are wins and there are glaring examples of why SDI is needed. Andrew Turner and I had the pleasure of meeting many of these folks in Nairobi, including a session with the Somalia Interagency Mapping and Coordination group (their walls were plastered with real pirate maps). Elsewhere in the UN, UNOSAT and GeoNetwork have supported GeoRSS for a while now .. using the simplest thing that works and not reinventing the wheel. There’s many encouraging signs.

The remarkable thing about the opening panel at UNGIWG 9, “SDI vs. Neogeography” was how everyone dropped the versus. Everyone in the room uses the web, uses google maps, facebook, posts cat photos. They get that web and bottom up standards work for easy interoperability, get the user focus, and some “excessive creativity” like maps of UFO sightings is a good thing. And the neogeographers also get that formal SDI and GIS have requirements of precision, complexity, and cartographic elegance that aren’t always met. So the two approaches are moving to adapt and hybridize and meet halfway. For forward thinkers in either camp, there isn’t an antagonism here, only an opportunity to improve things for everyone. Earlier this year, Chris Holmes gave a great presentation on this theme, the GeoWeb and SDI.

Nicolas Chevant and I presented the vision going forward for the UNSDI-t and how our work with OpenStreetMap supports that effort with real tangible results today. Nicolas has developed a widely accepted schema for the transport layer of the UNSDI, no small task when every agency had a different method of recording road data. The neat thing is that this top down process generated schema closely resembles the key/value tags in OpenStreetMap. Perhaps not so suprising since we’re talking about the same underlying reality, but instructive that two very different processes can get similar results .. the informal, bottom up crowd does work. And OSM can learn from the formal schema. In the UNSDI-t, the emphasis is on the practical passability of roads, rather than official classification. That’s important in many places that the UN operates, and in many active OSM communities. With OSM’s open tagging system, it’s easy to build equivalencies to other schemas and retain the original attiributes as well. So OSM has been able to bring in UN road data, and other thematic layers should pose no problem.

OSM has imported many data sets (AND, TIGER, GeoBase soon..), as well as UN data sets. Neil Penman, Brett Henderson, and Ian Checkley of IBM volunteered to start with the UNJLC data for Sudan, built shapefile imports, and tag equivalencies. Issues of data quality and provenance reared early, in an emergency, you scramble to grab any available data source you can find, copyright or not, but that won’t fly for OSM. The larger looming issue is data synchronization. OSM is a single node in a wide UNSDI, and the question is how do these multiple updating databases provide the opportunity for choice of best dataset and synchronization. It’s a huge issue generally for the GeoWeb, and OSM API v0.6 provides a little bit of the scaffolding OSM will need to accomodate multi-master sync. Overall Sudan proved the OSM and UN were entirely compatible mapping partners and set the stage for more.

After Cyclone Nargis, I met Christopher Tun at WhereCamp, and quickly got working to provide support for web mapping infrastructure in the relief effort. Unlike most every other disaster, the UN was not free to operate on the ground in Myanmar. They UNJLC did what they could with existing data and satellite imagery, and in a crucial change, the maps themselves started asking for contribution. Fortunately through Chris, we were connected to local engineers who could operate the server and survey the disaster zone. The team from IBM joined in again, and started working directly with Sahana, to deploy on a single server inside Myanmar, where it would be more accessible to locals. We set up a full OSM server and mapnik tile server running locally inside Myanmar, along with osmosis to help facilitate diffs and synchronization. Sahana integrated their excellent excellent GIS module fully with OpenStreetMap. We had great remote training sessions with the Myanmar team, in Bangkok, via the surprisingly useful webex, and there were plans for us to visit directly. However things have gone quiet since the flury of activity, and I only hope it was of some enduring use in the relief effort. In any case, there was good technical development and increased capacity on the tech support side of things.

Our last interaction was the response to Hurricane Hannah in Haiti. This time we partnered with CartONG, and specifically focused on getting a routing service set up with OpenRouteService. Chippy provided great assistance with Shapefile wrangling and presence in Geneva for follow up meetings. The OpenRouteService experimented with a simple OpenLayers interface for users to roughly mark blocked roads, and adjust the routing based on this quick feedback. The only downside was that our best data source was provided in a not entirely open form. We couldn’t import it directly in OSM, only into a special instance of ORS, and so we don’t have something of last use .. open data is always where we want to be.

All three of these emergencies provided opportunity for OSM and the UN to work closer together. My effort now is to establish links beyond the emergencies, in preperation, in the normal course of development work. The nascent idea is to set up OSM Corp, a group of volunteers working to build infrastructure, relationships, and processes for using OpenStreetMap in humanitarian work. The next time there’s an opportunity to respond, we can get rolling rapidly, and maybe even work towards deployment in the field with MapAction or CartONG. And wider, we’re open to collaborations with UNGIWG and its member agencies; this was my offer to the assembled…

Myself and others on the openstreetmap foundation board, and in the large global volunteer community, are keenly interested in the activities of UNGIWG and the development of the UNSDI. Data sharing among agencies, governments, and ngos for humanitarian purposes is totally complementary to the OSM mission, to create a free and open of the entire world. I whole heartedly offer our support and willingness to work with you all as an engaged and practical partner to build nodes in an SDI that works for the world.

UNGIWG, helping the United Nations to share its geodata

Last week I was in Vienna for the 9th plenary meeting of UNGIWG, the “United Nations Geographic Information Working Group”. That meant muted but real celebration of Obama’s victory in UN chambers. Good stuff.

UNGIWG is a group of UN professionals working together to solve problems and cooperate on all manner of things concerning maps and geographic data, most especially for humanitarian purposes. The UN publishes a huge number of maps, just check out ReliefWeb for a start; they utilize maps internally in the Security Council and Peace Keeping operations, just for a start; and have un-enumerated databases of geodata, of which Geonetwork is the tip of the iceberg. There’s a vast untapped potential for geodata and map sharing at the United Nations, to help the agencies themselves work individually and together in a better way, and to serve UN beneficiaries .. aka The World. That data is often locked up or slightly inaccessible or in weird formats, etc. The aim is to produce a “Spatial Data Infrastructure”, perhaps defined as “the technology, policies, standards, human resources, and related activities necessary to acquire, process, distribute, use, maintain, and preserve spatial data”.

To me that sounds like the same issues facing the Web at large, and IMO the Web is doing a pretty decent job at sharing. So with my long term interest and work with software in the UN, and insights into the whole GeoWeb thing, I was invited to participate, along with fellow “geographic hooligans” Schuyler Erle, Danille Nascimento, Sandra Sudhoff and Yann Rebois from CartONG, and of course my collaborator within the UNSDI over the last year, Nicolas Chevant. I personally ended up juggling three different hats, as OpenStreetMap Foundation Board Member, representative of FortiusOne (inherited through Mapufacture’s previous engagements in UNSDI), and finally UNDP, as consultant on WaterWiki and the UNDP Crabgrass install.

The final hat gave me a seat at the big table and closed sessions, open only to members within the fascinating and overwhelming alphabet soup of the UN System. The UN is in essence a diplomatic forum, and the corresponding ethos trickles down into the everyday workings inside the UN. UN meeting rooms have comfortable chairs and an equal playing field, and the organization of UNGIWG has produced an abundance of the necessary procedural documentation to get the officials of 30+ agencies working together and willing (potentially) to fund projects and send representatives and basically make the space for things to happen. Diplomatic procedures also produce long timelines .. we heard dates such as 2020 mentioned a few times. I’m all for the long now, but we all think that something can be done in a more web scale timeframe.

The common refrain was that “the technological problems are nearly solved, it’s the social process that’s in question”. This also extends to dealing with IT. While a majority of the attendees were technologists, they aren’t the ones in charge of deploying technology, and there are formidable barriers within the UN to doing something creative, especially with open source. IT sometimes treats its job more like a utility than a creative endeavor, and that’s a very limited view. I hope to raise this at the UN Web4Dev conference in February.

So that’s the stage, some idea of the issues and perhaps why our perspectives were invited into the show. More on what went down tomorrow.

In California and Support Open Geo Data? Oppose AB 1978

Update from Adina Levin: “I just heard from Sacramento — the bill sponsor has withdrawn the AB1978 and does not plan to resubmit it this year. In other words, the bill is dead.” Congrats people

When I relocated from the UK, my expectation was a welcome rest from struggles with absurd licensing schemes of government data. Well, Nope. Read this and act and write your California representative. More information here as well.

US Federal Law requires that all all works created by US government agencies are public domain, excepting classified material. This doesn’t require the works be released for free, that’s what the Freedom of Information Act instruments. But in practice, many agencies simply release data, resulting in primary public domain geodata sets like TIGER (the US basis for Navteq, TeleAtlas and OpenStreetMap), NGA Geonames (core component of Geonames), VMAP0, Landsat, and USGS Topos and Orthophotos. It’s a valid point that without a cost recovery mechanism, funds for maintaining and updating these data sets dries up, and the data quality suffers; this has been particularly true of the USGS. But I’d rather start to find solutions from the assumption of free and open to the public, rather than the other way around, and in practice the web and community of open geo data have stepped up to fill this role.

However this federal law doesn’t hold for state and local geo data. And as common in the States, there’s a whole spectrum of policies from public domain to commercial level fees. It’s almost the opposite of the UK, where national data is non-free, and local authorities would like to release their data, if only the OS wouldn’t claim derivative rights over that data.

Santa Clara County started charging high fees for distribution of its geodata. They were sued and California courts ruled that costs can only cover the cost of distribution. And really for digital data the cost of distribution is nearly zero. At the time this was going on I surveyed the terra nullius of new suburban subdivisions in Santa Clara and thought it a damn shame these data sources should disappear.

Well now, Assembly member Jose Solorio’s AB1978 is attempting an end around this ruling by extending the definition of exempt materials (usually classified public safety stuff, think precise locations of critical infrastructure) to “assembled model data, metadata, and listings of metadata”. Vague and ambiguous, the intended effect of this change would allow local governments to reinstate high commercial fees for access to their geodata. As stated here, “Rather than clarifying the Public Records Act, his bill’s proposed paragraph would make the Act more ambiguous, confusing, mis-informed, and obstructive of the public’s right to obtain its government’s records.”

If you’re in California and care about such things, write your representatives.

In a similar vein, I think about the trend of municipalities releasing their geodata for inclusion in Google Maps and Earth. In exchange for some small compensation and maybe some free copies of Google Earth Pro, cities happily give up the goods. Which is great, in part, as that data has been collected with taxpayer money and should be made available to the widest audience possible — and Google certainly has that. One town has gone all the way and has outsourced published all its geodata for collect Google actually, Jason Birch clarifies in the comments: they publicly published their geodata in KML, though orthophotos are only served by Google due to licensing restrictions . But but .. the data is still not truly free and available to any other company or hacker or activist to use. With disappointment, Google’s only motivation is to get that data into Google, not the public. The same is true of Google Transit, where special deals are made to release data to Google and no one else, something UrbanMapping is fighting the good fight on.

Working for government action at the national level is like moving a mountain range. There’s much more possibility to have an affect in your local government. Oakland Crimespotting motivated the Oakland PD to release their data in a machine readable format. Merano Italy released a treasure trove of CC-SA data. Inspire your local government with these examples and others, and push for truly free and open data .. leading to democracy, participation, and an inventive public life.