From Hacks to In-House, Flickr launches Geotagging. Tripletags are dead, long live tripletags!

Building photo maps, especially flickr, was one of the collective obsessions of Geowankers a couple years ago. Dav did the original awesomeness digging into EXIF headers, Trevor had some sweetness in 93 Photo Street, Mike Migurski cracked flickr with mappr, then there was fllickrmap, loads loads more..

I introduced tripletags to flickr with mapping flickr, an ugly but effective way to get flickr to do something it was never intended for. Flickr adopting maps should be an encouragement to everyone to continue to try and break things in interesting ways!

Dan Catt Google Maps, greasemonkey, crawler database extravaganza, Geobloggers, which impressed the pants off everyone, so Dan went to work to flickr .. moving the hacks in-house. I know everyone there has been working super hard on this, and the result is great, congrats. The clustering & mini photo browser is something I want to emulate in other maps.

They’re still working on the developer views/apis. I won’t have to predictably bug them about GeoRSS, since there’s already a clamor for it , and Dan confirms GeoRSS is coming. Dan also lets loose a currently hidden EXIF option, so flickr will work with great tools like WWMX Location Stamper, etc. There is no option to have flickr look back through your photos for EXIF GPS data (this would probably really grind their dbs to a halt), so I expect some more hacks to fill in this gap once the developers api is up.

Flickr is expected to respond to its hackers. Another launch today that caught my eye shows that responsiveness in companies you’d never expect. TeleAtlas is now requesting data bug reports from its users. The landscape is shifting around OpenStreetMap , and when behemoths start to move, who knows how radically the world will transform.

Another summer, another wiki! UN INSTRAW Gender Training Wiki. INSTRAW Tech, Take 3.

In what’s becoming an annual tradition, Anna and I have launched another Wiki within the United Nations, the UN INSTRAW Gender Training Wiki! This is intended as a resource for a wide community, and features such technical nicieties like integrated mapping and calendaring of all articles. I encourage you to check it out for yourself, but here’s a run down of how it went down, from my perspective.

The Conceptual Switch. This Wiki brought its own share of new challenges, as well as reinforcing lessons from the previous one. At the start, the success of the WaterWiki was convincing enough to pursue the Wiki route for gathering Gender Training resources (rather than the traditional Word Document favored at INSTRAW!) .. even if they didn’t quite know what they were in for with the Wiki. They also like maps, which have an even better integration in this wiki (more below). Anna organized two training sessions for the entire organization, introducing the concept and the techniques necessary for using it. We quickly simplified the interface of the wiki, and tried our best to bypass the nastiness of wikitext.

The biggest difference is the emphasis on connecting with external organizations, and multilanguage support. The WaterWiki was designed as a knowledge management tool within the organization; the Gender Training Wiki is intended for a wide community of organizations to access and share information on Gender Training organizations, events, and materials. It’s open for anyone to edit. And along with that, this Wiki, as it grows, will require ever greater effort to garden and organize the contributions .. an organizational effort that still needs to be met. Unexpectedly, the first external contributors to the wiki were men, listing gender sensitivity training for men. They intend to pursue the Wiki route with other projects, for instance one focused on Gender and Political Participation, and the MediaWiki is all set up in a farm mode .. this represents a fundamental shift in how INSTRAW approaches its mission.

Internationalization. INSTRAW is committed to supporting English, French and Spanish. But rather than launching three seperate Wikis, ala Wikipedia, we wanted to avoid duplication and patchy data and stick with one multilingual Wiki. The ideal solution would have different versions on each article in multiple languages; a user would be presented with the article in their preferred language, if available, with links to other languages, and default to English if not available. With limited time, I was unable to build an extension to do this, but think it’s possible. As a stop gap, all three languages are presented on the main page, and Spanish and French articles are categorized and automatically collected.

The structural text is translated, as per normal, but some of the extra features we introduced weren’t easily translated. Breadcrumbs, which has just a little traction as a concept in User Interface design, was completely nonsensical in direct translation to migajas de pan. Its meaning is only clear via a cultural reference to a German fairy tale, that’s not commonly known among native Spanish speakers.

Mapping and Calendering. Any page in the wiki can be assigned a location. This is done directly by browsing in a small Google Map, including location lookup with MGeocoder, inserted in the edit page. Rather than the difficult-for-the-novice method of explicitly typing coordinates using wikitext, or simple-but-not-intergrated use of a external service like Placeopedia. The editor never actually sees the wikitext that assigns location and inserts the map .. it’s all preprocessed using hooks, based somewhat on the MediaWiki Google Maps Extension (authored by Evan Miller, who I collaborated with years ago on some worldKit features). All of the geotagged articles are assembled into a GeoRSS feed, and displayed on the main page using MGeoRSS.

Similarly, any page can be assigned a date range; this is especially relevant for timetagging training courses. The interface uses the calendar widget from Yahoo, and all upcoming trainings are automatically collected and listed.

The technical details of the extensions and hacks, used and developed, are all listed for those interested. I haven’t released the new Google Maps Extension or the Calendaring Extension, but could be pursuaded to. I think there’s a lot to this approach, of adding to wiki interface elements besides the text edit box.

Our Guard at UN INSTRAW

Our guard at UN INSTRAW

The daytime guard at UN INSTRAW is about the nicest security guard I’ve ever come across.
After a hectic journey across Santo Domingo to the office, it was great to be welcomed with “Hola hermano” and a “Fight the power” style fist salute. The gate was almost never locked, just a chain hung across the gap symbolizing lockedness.
He’s on loan from the Policia Nacional. Next door to the INSTRAW compound is some type of military installation. Every evening the trumpet would bellow, traffic would be stopped, and our guard would dutifully lower the Dominican flag in unison our neighbors.

One morning he stopped me and tried to hand me some sheets of paper. It was some kind of multiple choice History test, in Spanish. Cuál era la causa de la guerra mundial una? a) los fascistas. b) revolución de los métodos de producción… . He wanted me to take it and look up the answers on the Internet. I didn’t take it. He asked everyone in the office. Later I learned the test was not his, but actually his brothers. No moral quandries for him whatsoever. A kind of beautiful Dominican attitude. We’re all in it together, we help each other, and if his brother can get an entire UN Agency to help with him multiple choice history test, then wow, he deserves even more credit for it.

Another week, he started saying everything in English. Good Morning, How are you, I’m fine. Practicing. I was happy to oblige, he obliged my own tortured Spanish. One week later, the English abrputly ended, and then it was just Spanish for the rest of the summer.

My sysadmin colleague at INSTRAW always wanted to snap a picture of our guard sleeping. He always woke up first. He may be sleeping, but his attention was finally tuned to any disturbance.

Hasta luego senor.

Image Overlays in GeoRSS

For the Mapping Medio Ambiente Workshop, I whipped up support for Image Overlays in GeoRSS, in MGeoRSS.

Here’s an example of Yahoo Maps overlayed on Google Maps Santo Domingo.

This is something I pondered before when working on imagery overlays in Google Earth for the Java Earthquake in June.

The idea is to set the relationshiptag attribute of GeoRSS Box to image-extent. The location of the image is specified in another media:content tag. The relationshiptag and featuretypetag of GeoRSS are meant to be folksonomic .. unintended ontologies and usages will be explored as developers work with the format.

<georss:box relationshiptag=”image-extent”>-70.01544 18.39777 -69.80567 18.563517</georss:box>
<media:content url=”santodomingo.png”/>

I’ll detail this usage in MGeoRSS next time I update there. The only additional requirement is including the excellent TPhoto extension.

This is a very useful way to specify overlays, and I hope to encourage other GeoRSS tools to adopt this convention.

Mapping Medio Ambiente

On August 22, in the newest building at the oldest university in the western hemisphere, a group of adventurous folk from the Dominican Republic Ministry of the Environment became Map Hackers at the Mapping Medio Ambiente Workshop.

Universidad Autonoma de Santo Domingo

In their work, monitoring pollution sources, preparing environmental impact reports, conservation, they generate large amounts of data, GPS waypoints, imagery .. but generally lack the technical and financial means to organize this information in a traditional GIS. Mostly GPS readings are just noted down in paper notebooks. Clearly more could be done with this data, further internal and external communication and organization with geo-visualiation and -analysis. The idea for the one day workshop was to introduce the web/hacker oriented approaches, to drive use of the great geospatial tools from the bottom up.


We had a large agenda, though the day ended up looking a bit different. Our main activity of the day was production of a geotagged photo map. In three shifts, groups went out with the GPS and digital camera, checked that the clocks were in sync, and took a stroll. Back in the meeting room, I demonstrated how to use GPSBabel to pull out GPX (and KML), and used WWMX LocationStamper to geotag. Then upload to flickr, and display on a little custom mapping flickr hack (using flickr in preperation for their native stuff). We then distributed the GPX track and photos to each person’s computer, and they went through the process of stamping, uploading, and mapping themselves.

From the feedback I got, this simple activity was really inspirational. Within a few hours, they went out and physically collected, transformed, uploaded and visualized data. Making flickr maps itself may be useful in their work, but more widely they now understand that they personally have the ability to work with geospatial data and maps.

At the end of the workshop, we distributed copies of Mapping Hacks, Google Map Hacks and Web Mapping Illustrated to various departments within the ministry and donated a set to the university library. The participants were very happy to receive these, and it should provide a foundation for further explorations. There were one or two folks with a bit more experience with GIS, so Web Mapping Illustrated will be on target for some. The IT staff as well will find all these useful. Big thanks to O’Reilly for the donation.

Mapping Medio Ambiente Workshop

Naturally, not everything went to plan. Partially this was technical .. I didn’t get a chance to test things out in the meeting room before, so lots of time taken up getting the correct gateway for the wifi, and a fruitless attempt to use FTP, blocked by the university’s firewall. The same “everything will be fine” Dominican attitude which stressed me out in preperations for the workshop, served well during the disruption. The unexpected is expected. People browsed Google Earth without any prompting, and probably could’ve continued all day. The Directors of the Ministry and representatives from the University helpfully gave speeches while we scrambled to get Wifi working .. the speeches were planned anyway, a chance for me to practice my Spanish comprehension.

But also I overestimated what we could accomplish in one day. It was probably good that FTP was blocked, since teaching just this basic concept of moving files from the computer to the server could have taken hours. And in retrospect, my plans relied on building custom hacks rather than primarily on many of the established web services for building maps. I think eventually the Ministry will build capacity to publish KML, GeoRSS, and Google Maps on their website, but in the mean time its enough to be able to useflickr and Wikimapia and Platial.

Wikimapia was really really intuitive for the participants to use; I think more of the social mapping websites, and the web in general, should strive for this simplicity of interface and functionality. Platial, while definitely useful for creating personal maps, felt overburdened with functionality at first impression (I have a bunch of feedback for Platial when I run into them again). Watching a novice user, especially a non-native English speaker, foreign cultured, trying to navigate Web 2.0 apps in really telling. All the things I love about Web 2.0 — the irreverant attitude and language, the ever changing plethora of features, playful design — are major obstacles. Errors are part of the game — in order to use the web, in a way you have to understand how it can break (anyone who promotes Firefox will have to explain that some sites only work in IE, HTML incompatibilities, etc). Imagine if your refrigerator broke down 10% of the time you opened it to get some food? What if flickr had been having a massage during the workshop?

I really had a good time doing this Workshop, and hope it catalyses further mapping work at the Ministry. There’s so much opportunity to adapt the web, hacking, geo- in the Dominican Republic, and throughout the world .. its a shame I’m heading back to the UK now. I was entreated to stay, or come back again, and I hope I get a chance soon.

INSTRAW Tech, Take 2: CiviCRM

There is (soon “was”) no unified contact management system here at INSTRAW. People employ Outlook Contacts, Thunderbird Contacts, Excel Spreadsheets, Word Documents (one with over 3000 contacts!), Custom Web Databases, Business Card Scanning Software, and yes, 3×5 paper card files.

For any organization, but especially a small, donation dependent, research oriented organization like INSTRAW, a CRM is indispensible. Pinging everyone in the organization when looking for contacts slows the whole organization down. For sending invitations to the upcoming film festival, contacting media with press releases, hitting up donors for donations, ordering toner. It may not be the sexiest application I’ve worked on this summer, but I have no doubt that it will make the biggest improvement to operations here.

When this requirement was first floated, we went out and evaluated our options. Commercial, open source, or maybe even develop something in house based on CakePHP. Finding CiviCRM clinched our direction. Free, open source, a distinctive pedigree, with connections to CivicSpace, and led by Don Lobo. I worked with Lobo at Yahoo, and his presence on the project made me confident that it is a solid piece of software, which will be supported in the future.

Of the three major techs deployed this summer, CiviCRM was the one I had to worry about the least. So much could be configured and customized through the interface, that for the most part our system administrator took over deployment. Of course, there is plenty of opportunity to contribute code, and make this workhorse even better, particularly some advanced GIS functionality, but only so much time…

Really the major challenge will be import. With so many different formats and users, it will be a challenge to get it all input, rationalized and verified. But once that painful social process is done (probably by one of the intrepid interns), it’s done.

Mapufacture, update

Mapufacture has just been updated with a shaper look, some new features, and some backend improvements.
Credit to Andrew Turner and Guilhem Vellut, whose incredible map & rails hacking skills, and great enthusiasm, are speeding Mapufacture forward. Thanks dudes.
Thing is, we’ve been meaning to hold off the release until a bunch more cool things were coded up, but with all the rails security ruckus, the easiest way to upgrade rails was going to be pushing our latest code. There’s still a bit of polish left to apply, and our roadmap takes in that bunch more cool stuff I just mentioned, so keep watching.

My favorite improvement won’t actually be visible to users at all (yet). Guilhem linked together Mapstraction with Mapufacture, by releasing the YM4R/Mapstraction Rails plugin and integrating it in. This gives us the potential to do some really neat stuff. And this isn’t the last amazing integration among our mapping projects…

INSTRAW Tech, Take 1: CakePHP

UN INSTRAW researchers have become fond of building databases of their research, for public consumption .. very simple, small, searchable lists of research institutions and documentation and the like. The permanent web dev here, being a very nice and agreeable guy, has allowed the data for these databases to be compiled in Microsoft Word .. which he has dutifully, if not a bit pervertedly ;), processed into SQL INSERTS using a combination of AbiWord, yacc (!!), and Perl.

One of my tasks in taking on part time work here (in exchange for salary, and office space for my own work, in the other-part time .. well you can imagine how that turned out..) was to build one of these databases, for Gender Research Methodology references. Clearly there must be a better way! Even with the perverse call of yacc in my possible web development toolbox.

With only a single table, the most braindead CRUD scaffolding was going to get me 99% of the way there. The Word Documents would be banished (except for one final import for this database .. which became so painful, battling AbiWord’s crummy character encoding translation, that I abondoned the automated import for YES a manual import)! I started looking at Rails obviously, but thought again, this place being by and large a PHP shop, I shouldn’t foist too much new stuff on the place.

Looking around for PHP Frameworks, there were quite a few, like Zend and Symfony .. but CakePHP kept jumping out at me. Maybe I was hungry, or maybe it was the way the logo danced on the CakePHP site. Being called the Rails of PHP was somewhat hopeful, and going through the 15 minute Blog Tutorial sealed the deal. The architecture and conventions were dead on similar to Rails. Sure it didn’t have all the features of Rails, but in a way it felt a little simpler to get around (and maybe not so many panicky bug fixes). PHP still feels comfortable to me.

The resulting application is dead simple — CRUD administration, with a Searchable front end, on a single table. I basically only added a Helper for what I termed a Psuedo-Set — a group of Boolean SQL columns that share the same prefix. I wanted an easy way to display these as a group, and wrote a helper to do so based transparently on model inspection. (If this is of any use to anyone, let me know and I’ll publish it). Also, CakePHP’s builtin TextHelper had slightly buggy autoLinkUrls method, so I wrote my own version.

The next time they need a database app here, it should be possible to be up and running within a day. The INSTRAW Gender Methodologies database may not be that exciting technically, but it’s possibly the first United Nations installed of CakePHP.

I’m pretty impressed with CakePHP. Well done Bakers! And I can say, the folks on #cakephp were incredibly helpful (more helpful than #rubyonrails) and humored my mostly dumb questions. If PHP is your thing, and you don’t want to enter the Ruby world, check out Cake PHP.

INSTRAW Tech Showcase

The summer here in Santo Domingo is wrapping up. There’s been a lot of adventures, and a lot of hard work too. Among many things, I’ve been working part time at UN INSTRAW, filling for the web dev who’s taking some time off. It’s been cool to again work at the United Nations, this time at a much smaller agency .. which has meant more freedom technically and more long term impact, I reckon.

I’ve introduced three major software packages here, and done a fair bit of hacking on each .. MediaWiki, CakePHP, and CiviCRM. I’ll be writing some reviews of how its gone deploying and programming within these great pieces of software, starting off with CakePHP…