Archive for June, 2005

APIs, APIs, APIs, and the open geodata community

APIs, APIs, APIs, and the open geodata community

Google Map API. Yahoo Maps API. Microsoft loves user created content. Everyone loves users and developers. They big corps seem to surface realize the utility of what geohackers have been discussing and building, and are ready to role it out masively. That’s cool, right.

The point of geocoded RSS is to leverage the architecture of creation and aggregation tools out there. By encoding annotations in RSS, widespread ease of use leads to an emergent layer of open geo data that is searchable by location and text, taggable, ready for repurpose.

Yahoo Maps API accept the geocoded RSS “substandard” (help please OGC) for building maps, and that deserves an exclamation mark! It’s a good start, addresses a certain segment of usage, and spurns geodata creation on the network. Now Yahoo needs to start aggregating these feeds, and build a search interface to user created data in Local. Apparently I still work for Yahoo (w/o the benefits ;), if you guys want to do something, let me know. Or how about map links in the My Yahoo RSS modules? That My Yahoo Maps module is looking a little crufty.

Criticisms: the map is still kinda pokey and RSS feed errors are not well reported, please give some feedback. Not thrilled with the “ymaps” namespace .. “Address”, “CityState”, “Zip”, “Country” all have general utility beyond Yahoo, and this should be pursued in open standards. “Groups” can be handles by <category> or <dc:subject>, there’s no need for new fields here.

Google Maps API is aimed at developers. There’s no portable annotation format specified, it’s all through Javascript calls. That’s fine, leaves it open to apps build on the platform. And leaves the creation of community open to other apps. An excellent Google Maps Hack would be a geocoded RSS parser/translator to Google API calls.

Virtual Earth has professed user&developer-love, but it’s yet to be seen how easy it will be to get information in and out of their system. And it’s very much about community .. will users feel that the system is somehow their own? flickr lessons.

Now that the tools and formats are widely available, the next challenge is to make it easier for users to geotag, aggregate and search.

Comments

Where 2.0, Yahoo Structured Tags

Where 2.0, Yahoo Structured Tags

Starts in about an hour. Demo’ing tonight. I may try and take some notes here, who knows if I’ll get a chance.

The big news right now is Yahoo’s Social Search. This is interesting on many levels, but stand out for me is

Structured tagging – The internet is about much more than web pages – key dimensions like time and location can be as important as the content itself. With user-provided structured tags like “geo:[location]” applied to pages, search results can now can include maps to locations in addition to the web page.

Yahoo supports geotagging! and other structured tags. [location] right now refers to text for search in yahoo maps, hopefully lat/long will be coming soon

Comments

Microsoft RSS

Microsoft RSS

It seems that by writing about it, I’ll perpetuate its importance, but anyway, on RSS RSS Support in Longhorn I think they miss the point of the strength of RSS: simplicity, and power. Their simple list extensions don’t pass the ugly test, there’s much more succinct ways to express these things

Comments

OSGeo Postprocessing

OSGeo Postprocessing

It’s time to put down OSGeo impressions before it starts to fade. Very good conference on many levels, learning loads, meeting formerly virtual people, and new people, spreading a bit of worldkit-georss gospel. worldKit made it on the tshirt :). Overall there is a great sense of community, and great sense of happiness in this work. Buzz. Geospatial technologies are at this intersection of technology, science, politics, and the natural world that just make it the right place to expend useful energy.

First off, O’Reilly had a good showing, met Simon the editor, the authors (in a new locale), and actually held the book in physical form. Met author Tyler Mitchell of the other book (will need to pick up this one too) and Nat Torkington, putting the finishing touches on a promising Where 2.0.

My appetite was primed to get more involved with projects like mapserver and GRASS; I’m just waiting for the excuse. Part of my presentation, I discussed integrating worldKit and mapserver: from discussions afterwords, it’s clear that the way to proceed is to learn mapscript to build the tie-in. It was good to learn that ECW is supported in GDAL since I have a project with this format that’s slipped by. On GDAL, we’re talking about adding a driver to support geocoded RSS (something I’ve done by parsing ogrinfo output, but this would be tighter).

GRASS engages the analysis portion of geospatial, while I’ve focused on an approach to spatial data infrastructure and visualization. Marcus Neteler keynoted on the history of GRASS (another military project dating to 1984) and its capabilities. The case studies were what engaged me. Every step was fascinating, from data acquisition, analysis, visualization, and working outcome, so I’ll go overly detailed. To mitigate collisions between vehicles and deer in Italy (wildlife is classified as state property there), they identified deer migration corridors and high density accident areas by analysing road curvature, 25 variables of land cover, terrain complexity, and traffic data from sensors, with support vector implemented in GRASS and used to derive 4 risk classes, leading to signage and underpass mitigations. Machine learning within GIS, awesome. The other case study, on unexploded WWII bombs dropped in Italy and still undetected, analysed military attack reports from the US/UK, terrain (bombers sometimes had to unload just to get over mountains), and crater detection from aerial photos. Combined with current census data, they could draw up costs and evacuation plans for likely bomb sites.

From working on integrating WMS sources into mapufacture, I’ve become interested in techniques for dealing with these huge data sets. Schuyler laid out his methods for setting up the vmap0 WMS. The key seemed to be Stephen Woodbridge’s shp2tile and ogrtindex, which pyramids vector data in a similar way to prepared raster sets. Vector generalization is also key, for simplifying unnecesary complexity in Tiger (Douglas-Peucker and Grid-Based algorithms) (and I’ve been interested in this for preprocessed flash representations of vmap0). Schuyler’s idea for a distributed WMS cache, for things like Landsat7 was well received(JPL could send astronauts to the moon, but can’t keep a WMS up and running). Any sort of caching is a huge improvement, Stephen implemented with mod_rewrite (and as Anselm implemented with squid for civicmaps). The developer of Ka-maps talked a bit about their demo, and relevant problems with label edge buffers and repeated labels (perhaps solved by creating larger “meta” tiles).

There’s very healthy competition in the visualization space with mapbuilder, kamaps, and worldKit. AJAX is obviously hot, though I still think there are some advantages to Flash, like real vector drawing and smooth image manipulation (which I laid out during my presentation following a Flash-hatedown in the previous talk :). Interestingly, in Dirk’s closing keynote on the Apache Project and lessons for Geospatial, he sighted the core difference being that GIS is visual, closer to the user, and solution oriented .. less a utility like a webserver, more motivated by “visionary” led development, in an unsettled innovative arena, so many features of Apache’s processes don’t necessarily apply. Perhaps this is why mapserver is so well developed, while open source interfaces still lag behind. Think there’s also something about the data that differs between the geospatial and traditional web; interoperability is a core motivation. Still the no agenda affect and cost benefits of sharing vs hoarding are usefl ways to examine projects.

One thing I had to repeat often was that worldKit was not actually open source, but did interop with open source projects and supported the creation of open data. Surprisingly, people were not as open source fanatical as I presupposed; everyone is sympathetic to the need to eat and understand you need to find the right model for sustainability. Many companies present contribute to open source software projects as part of work for clients, or within research. I am left with a lot to think about regarding future directions.

There’s also great interest in making these tools accessible to non-experts. Google Maps, and hacks, were mentioned many times at the conference, in a its changed all expectations sort of way (makes me wonder if mapserver will be a hot topic at Where 2.0). Though there’s still some conception of “barbarians at the gates” in the People’s SDI, there were plenty of examples of efforts that I think get it. HostGIS provides a hosted service for web based GIS, with powerful data integration and display options encapsulated in useable form. They’re acquiring openly available geodata from many sources, and making available for remix by their clients (like a heavy duty mapufacture). GeoVista Studio is an interesting “codeless” programming environment, using wiremodels to construct GIS data and functional vlows. A mapserver based US Census data explorer featured many powerful features, as with a similar project in the UK.

One block to widespread geodata creation and sharing, has been the overly complex OGC standards. Even there, almost everyone rolled their eyes at WML. To get real adoption among non-experts, standards need to be at an understandable level. This is geocoded RSS and similar simple standards. There looks to be interest in making simple geoformats interoperable with more complex OGC standards. This could be just the thing to get the ball rolling. Jo is collecting prior art here. Microformats are gathering inertia, and there are many similarities with geotags .. an evolutionary, useable today format. I’m surprised a Geo microformat hasn’t come out yet, will have to remedy that.

Now, what will happen at Where?

Comments

Sunrise on the Everglades

Sunrise on the Everglades

Down in South Florida after OSGeo, visiting my Dad and Grandma. This part of the country has some of the most spectacular sprawling exurbs and wide ass roads. I spent part of my childhood down here, and used to visit my grandparents here in Hollybrook back when it was in the sticks, a retirement oasis among endless blocks of cattle fields and pine trees. Familiar bleached-car-centered-single-family-home-big-box-shop has enveloped this place. Fortunately there’s the semi-natural Everglades barrier providing some end point to the American Dream.

The cities of South Florida may be huge, but the Everglades is huger. Most folks are just dimly aware that it’s there, and used to be right here before massive hydrological projects made building here possible. It’s not that far away really, and you can easily get out there on an air boat. I almost think it should be ecofascist compulsory to take a trip out there and get some awareness of the natural world which seems so far away.

My Dad and I got up early this morning, hit the Waffle House, and drove to where endless development ended, giving way to mega-churches, plant nurseries, and finally nothing at all. Rented a little motor boat from the Everglades Holiday Park just to take a look around, which is slightly unusual to the leather skinned old timers running that place. It is canals around this point and not proper Everglades, but that doesn’t stop tons of egrets, herons, cormerants, and a cardinal from hanging out. Zipping down the canals, sky reflecting on the water, like flying between two worlds. Found two alligators, scaring one into a dive with a big splash. The highlight was spotting a bald eagle perched on a dead tree, taking off with a fish in its talons. Here’s our tracks on mapufacture (lines aren’t yet supported on mapufacture for arcane reasons). No wildlife pictures, did get a shot of Dad, he’s easier to capture.

dad in everglades

This was my second outing on the Everglades. A couple years ago, Anna and I canoed in Everglades National Park, in the Flamingo area, where the definition of land, river and sea blurs gradually. I hope to spend more time in the Everglades, it’s a unique place, deserves hydrological and ecological restoration. Work on developing spatial ecological models in the Everglads was some of the first work on “computational ecology” I came across.

Comments

OSGeo Presentation

OSGeo Presentation

Gave a presentation at Open Source Geospatial. on worldKit and integrations with WMS and mapserver. It went pretty well, giving a real alternate approach to geospatial. The talk was primed with a presentation on GIS for non-experts, and a heated discussion on Flash! Thanks to the organizers for inviting me to present.

Here is the presentation for download.

Comments

Developing Mapufacture

Developing Mapufacture

This article was published today on O’Reilly Network, digging into how mapufacture came about and how it works. Great! Part of a series of articles promoting Where 2.0, the end of June.

I’m going to be there. Taking part in the Demo Fest, where apparently undiscovered entrepenuers are discovered. Bring it on!

Comments

Arrived in Minneapolis, OSGeo

Arrived in Minneapolis, OSGeo

Long journey from the UK. There are Americans everywhere! I’m starting to feel like a foreigner.

I’m here for Open Source Geospatial ’05! Presenting on Friday, on worldKit and some integrations with mapserver, and other stuff generally. Great! I’m excited!

From my hotel room, Minneapolis looks like one of those American towns that appears much bigger than it really is. It’s like Las Vegas casinos cities have distorted the scale of real cities. I think most of the population is at the baseball game tonight.

Comments

Pessimistic Software

Pessimistic Software

Matt Webb writes on how the Web is San Francisco circa 2001, writ large. Place, culture, and infrastructure has a much greater pull on the kind of software produced than we imagine, and the Web has only reinforced that. Software is intrinsically optimistic, in a California accent.

Crackers predominate in former Soviet states with intelligent trained programmers and black economies. India gathers outsourceable software tasks. Online gambling is built from financial black hole island nations. Matt himself is developing evnt in London, with broad horizons but initially focused on the anarchic London media arts community.

2001. The thing about San Francisco is it’s incredible dynamism. SF today is a very different place than from the boom (damn thankfully!), and was even more different in 1995. The city was destroyed by fire in 1906, reborn quickly, and has continued in a cycle like that since. The cycle has charged up so much, SF heads to Burning Man every year to compress city creation and destruction into a week.

On transport and online delivery services, I think that may have more to do with the great influx of new people during that time. Public transit in SF may be worse than London, but driving is just as bad, and the newcomers were maybe used to a more auto centered lifestyle. Without the time or inclination to explore their new surroundings, online delivery was the perfect solution for an internet carpetbagger lifestyle.

I used to ride underneath the Central Freeway to catch Caltrain down to Yahoo in Santa Clara. Homeless encampments used to blossom and fade down there periodically, a fractal reflection of SF itself. One evening, my Schwinn Suburban got a flat right near a few tents and I had to stop. A junkie jumped from a tent, accusing me of stealing something from him, and it started to look like I might lose the laptop over my shoulder. I raised the bike Critical Mass style as an impromptu defense, and we had a standoff. That was an archetypical moment, encapsulating all the contradictions in boom town SF. And in myself.

Anyway, my possessions and I got out of there okay, without any violence, thanks to the intervention of a passerby, who could have easily joined in against me. Thanks man.

It made sense that virtual reality grew someplace between psychedelic SF and numbing suburban monotony of Silicon Valley. At the height of the boom economics had pulled hundreds thousands of minds, tapping their energy just like those Matrix pods, all for the process of creating virtual worlds so much more enticing than Highway 101. Something like this prompted me to move here to the UK five years ago.
And that’s reflected in my software too. worldKit and mapufacture are tools with a global focus, designed to be used in ways I never envisioned.

Though I’m still surprised that any auto journey is going to take twice as long on UK roads than US freeways.

Comments

Google Cease and Desist

Google Cease and Desist

gMerge has gotten the take down. I received nearly the same cease and desist email in February for the google maps tile grabber. The difference in mine was use of the word “scraping”, rather than “using” (guess it’s a friendlier word?). I had been wondering if Google had just given up after the proliferation of gmaps hacks. Now it looks like the line is being clearly drawn. Assembling google map tiles is a pretty easy thing, expect to see this pop up again.

Anyway, it’s not even necessary to scrape Google. For freely available imagery, MapProxy can build maps worldwide with 30m resolution, and in the US at just the same resolution as gmaps.

Comments

« Previous entries Next Page » Next Page »