Doing

The past few months have most entirely been about discussing, strategizing, arguing, planning. Really missing actually doing stuff.

So I decided to ignore the emails, concepts notes, budgets, design documents. Build something. Just small things, but cool things. Makes me happy.

osm-changesets

On the OpenStreetMap site, the history tab is super important for keeping tabs on changes, but visually dull and hard to immediately parse. I added a map, with highlighting between the list and map views. TomH cleaned up the code and improved the layout. Basically, just some javascript and view changes, and I had to get my OSM installation up to date. Needs some tweaking, but a good start.

I was pushed to do this by the Strategic Working Group. We’re discussing the OSM front page and usability, going in circles, and realzing the only way forward was to just do some things, however minor, to iterate our discussion.

Looking at the history tab, I realize there’s many ways to go in the realm of monitoring change. Make monitoring more interactive, customized to the places your care about, and the mappers you trust. There are a whole set of external monitoring tools and perhaps some of the tools, or the ideas, could be integrated.


Kibera-1961

That is Kibera in 1961. Choose “Nairobi 1961″ from this map. Really, just a preview, it’s going to blow people’s minds, especially with Brian Ekdale’s concise history of Kibera.

Last year in Nairobi, I became aware of an amazing archive of historic aerial imagery, flown by the RAF, acquired over all the British Colonial possessions, several times over decades. The images were used for creating ordnance survey maps … but since then the original images have been sitting in various dusty basements, most recently Oxford. Paolo Paron has a plan to get it all openly online. But in the mean time, you have to go to Oxford, sort through boxes of square images, and request photographic copies. I had the chance to do this back in December, and it was wonderful. 1961 is 50 years ago, meaning these images are out of copyright!

I receieved from Oxford eight largish tiff images. Seeking first to rectify them, I tried various incarnations of map warper (including a local version) but with 30 MB images, it was choking. Next I tried to find a desktop option, and found good results with QGIS georectification plugin. I stitched together 6 of the images, roughly (if I spent more time, would’ve adjusted the different brightness levels and try to align better the two North-South runs that covered Nairobi, one at a slight angle to another). Then loaded a shapefile of Kenya OSM into QGIS, set control points, and warped. Took the resulting GeoTiff onto the generous resource of hypercube, warped the image to spherical mercator using gdalwarp, and after some hints from winkey and crschmidt on #telascience, “gdal_translate -expand rgb”, to take this single band image into RGB, which TileLayers seems to prefer.

There’s a lot more to say about this image, and that’ll be built up in the upcoming new mapkibera.org site. And of course, there’s those 1.5 million time capsule images sitting in an Oxford basement to open up.

Flow-Liberia

Friends of my mine from Nairobi, working for the World Bank, undertook a substantial data collection of all water distribution points in Liberia. Pretty interesting compare and contrast with community mapping. I was pinged about this on twitter by Ned of Water for People, who helped develop the Android/Appspot application for this data collection, called FLOW. My immediate response was that this should be open source and open data, so other people can do cool stuff with it. Max took up the conversation and obliged (as much as he could, the data license is still under discussion). So I had to respond and do something cool with the data.

The flow web app is ok, but a little clunky. It uses the Google Maps API, and with nearly 1000 markers, it loads slowly and interaction is tricky. So I wanted to try a tile-based approach, to improve the speed and visualization, and this seemed like the best opportunity to get into TileMill, part of the MapBox toolset by Development Seed. After going through the tools, I am extremely impressed with what DevSeed is building, and hope I can help out by finding good uses, digging up bugs, and contributing here and there.

So I installed TileMill, updating mapnik in the process. Really nicely designed app, with laser focus on building nice tiles. One of the amazing features is map interactivity, which easily and efficiently turns input data sets into mouseover and infowindow bubbles. The trick it uses is grid renderer, Dane’s extension to mapnik which builds tiled json files, used by the wax library for OpenLayers (and ModestMaps, and GMaps) to define hit areas. Basically, much more efficient than creating markers.

Finally, Dane showed me gridsforkids, a simple sandbox for building image and json tile sets for standalone, offline map apps. I had to grab the “.mml” file created by TileMill, have carto convert that to mapnik xml, and then use the custom generate_tiles.py to build everything. Found a small bug in how TMS is handled there, and had to hack in a few things for my data set, but it basically works (a few problems with the hit areas, still investigating). One way this could go is streamlining the process, where one of the outputs from TileMill was a tar ball bundling all the tools to run the map offline.

6 Comments

  1. New Changeset display | OpenStreetMap Foundation Blog said,

    May 6, 2011 @ 1:12 am

    [...] Read more about this from Mikel’s announcement on his blog. This entry was posted in Uncategorized. Bookmark the permalink. ← OpenStreetMap tile layer guidelines [...]

  2. Emmanuel Kala said,

    May 6, 2011 @ 1:19 am

    Good stuff as always Mikel. Seems you’ve got quite a number of hacks under that beard!

    I came across the Liberia FLOW project sometime back and I’m wondering if you have any leads on anyone who has a similar dataset (water distribution points) for Kenya.

  3. mikel said,

    May 6, 2011 @ 1:50 am

    Ask our friends at Upande, they might have some leads.

  4. b42 said,

    May 6, 2011 @ 12:39 pm

    Great work with the history tab, it looks really useful!

    Unfortunately, the tab is still rather useless to me — most of the edits shown do not concern any point in the current rectangle (but are listed anyway, probably because the area in which they were done overlaps with the shown area). In some previous version, most of such edits were marked as “big”, which at least gave some clue that I should ignore them. Would it be possible to only show edits that actually touch some point in the currently shown area, or at least filter out the “big” edits? (The better question probably is, where should I complain about that?:)) Example of such an area: http://osm.org/go/0Jv1v6V1

  5. mikel said,

    May 6, 2011 @ 3:25 pm

    Yes, known problem. The issue I’m told is that filtering out “big” would be pretty inefficient, and the better solution is to integrate OWL … but that’s been stalled for about a year. A friendly ping on the dev list on this issue couldn’t hurt.

    In the mean time, I’m going to look at adding the “big” marker back in somehow … it was removed to save space, but I think we need something like that for the time being.

  6. Harry Wood said,

    May 19, 2011 @ 2:01 pm

    @mikel yeah. someone was asking about the “BIG” marker on the foundation blog here

    @b42 Yeah this is a much complained about problem. See also: How do you display only local edits in the changelog? and How do I see the history for my area?

RSS feed for comments on this post · TrackBack URI

Comments (6)