Mike Blumenthal article on Google using its own geodata

Informative article on Google using its own data by Mike Blumenthal at Search Engine Land

Gamaray AR Browser Open Sourced

Development on Gamaray the AR browser stopped on September 30th.  Today, the Gamaray source code was released as open source and available for download.


In light of Layar wanting to charge content developers then changing their minds for the immediate future its nice to have alternatives.

Update on Dell Mini with OSX

It's been six months, so I thought I would do a quick update on my Dell Mini.


What works

• It's small and light weight: I travel frequently and its nice to be able to stick a notebook in my purse/backpack/grocery bag.  In contrast, my MacBook Pro (the old 15 inch model) weighs around  7-8 lbs with the power supply and requires a laptop bag.
• It's powerful enough: I typically have 10 -15 apps running at the same time.  This includes MS Office, Eclipse, Apache, Postgres, a couple of browsers, Thunderbird, Colloquy, Tweetie, Evernote,  Python stuff, java web apps, text editors - the list goes on.  If things go pear shaped, it's usually a browser hogging all the memory.  I recently built the GeoTools library and I think the Dell Mini did it in under 20 minutes.  I would have never believed that I could do my typical workflow with a 1.6 GHz processor with 2 GB of RAM.
• 32 GB of disk is sufficient: I thought that 32gb would be cramped, but I have yet to run out of disk space.  I don't think I've dipped below 5gb of free disk space, but I do cheat a little and use the 16gb SDHC card for storing big files.
• Wireless works everywhere: With my MacBook Pro, I have dead spots in my house.  This is understandable because my house is old with brick walls and with non-load bearing walls covered with plaster over a wire mesh.  I don't have this problem with the Mini, I get WiFi in every part of the house.
• Decent battery life: I can easily squeeze out 3 hours of work using the standard battery on a flight between San Antonio and DC, which is a 3.5 hour direct flight.
• SD card:  Being able to copy photos from the SD card on my camera to Flickr or just on to my drive is teh awesome.  No additional cables needed.
• Fast boot: So I have to hard boot on occasion, especially if sleep decides to throw a tantrum.  At least I don't have to wait very long, usually under 20 seconds.

What doesn't work

• Form factor:  Its friggin small, so I use a USB keyboard, a wireless mouse and an external monitor to be productive.  Using Spaces to organize the workspace sort of helps, but nothing beats a big screen.  I've gotten used to the Mini keyboard, but I still prefer the Apple external USB keyboard when I 'm  working.  I installed Melkort's touch pad drivers, but the gestures are not the same; i.e. one finger scrolling on the mini vs 2 two finger scrolling on the MBP.  The keyboard and external monitor are really helpful, but I can work without any of these things, if need be.
• Sleep:  Sleep used to work flawlessly, now it can be a crap shoot.  Sometime after the 10.5.7 update sleep started acting up.  Not a big enough deal to spend time tracking down a fix though.
• Watching movies: I've tried several versions of VLC and ran through all the fixes in the forums, but it still has halting problems.  Also sound is bleh at best, even with headphones or external speakers.  Really harshed my nostalgia when watching Space Above and Beyond.
• Monolingual:  Using Monolingual to slim down the install proved to be a case of being to clever for my own good.  Turns out that I did need some of those printer drivers and superfluous architectures.  I spent a couple hours debugging a make file before I realized that I only had an i386 architecture and that I need to remove the other architectures from the make file.
• N270 processor does not support virtualization:  This is the biggest thorn in my side with no work around.  More often than not, I have to look at stuff on a different OS; so  I use Virtual Box (gave up on Parallels) on the MacBook Pro.

For what it's worth, I haven't applied the 10.5.8 update but apparently my 10.5.7 update instructions work.  I did upgrade to the dellefi1.2.5a which got Melkort's touch pad drivers to work. Other than that, there have been no changes to the system since the 10.5.7 update.

I've use the Dell Mini as my primary machine for six months under heavy use.  My 15" MacBook Pro is used mostly for running other operating systems and watching movies.  Given that the MBP is 6 times more expensive than the Mini in its current configuration, leads me to question the conventional wisdom about buying the most hardware you can afford.  I think that this goes to show the OSX performs quite well with modest hardware.
   
So far, the advantages of portability outweigh the Mini's shortcomings.  However, that hasn't stopped me from eyeing the 13" MacBook Pros, which would do everything I need, just not in such a small package.

How To: Jars in a jar using Eclipse

I wrote a utility using GeoTools which uses a couple classes to transform geometry from one projection to another.  Projecting geometry uses EPSG definitions stored in the gt-epsg-hsql-2.5.7.jar which does not get bundled into an executable jar; for my utility to work as executable jar I needed to include the gt-epsg-hsql-2.5.7.jar.  


There are several approaches to building a jar that contains jars.  They typically involve using ant to bundle the files into the jar and a special class loader.  Writing the build.xml file and the class loader can be tedious.  Fortunately, the Fat Jar eclipse plugin simplifies the process and includes the One-JAR class loader.  


Installing the plugin can be done using Software Updates:

Click on Add Site

Enter the web site URL

Select the Fat Jar plugin and click on Install

Click Finish

Restart eclipse to finish the installation.

To build your project with external jars included, click on Export

Click on Other and select the Fat Jar Exporter

Select the project

Enter the name of the output jar, select the main class, and check off the One-JAR option to use the One-JAR class loader.

Clicking on next opens a dialog to select the jars to include, the default selects all external jars.  Unlike a typical executable jar, all the imported classes are not included so take the default and click on finish.  The Fat Jar plugin will build the jar with the external jars included, note that the final jar can become quite large.

Flying with a cello, part 2

On my return flight, I arrived at the airport three hours early expecting more ticketing fun and games.  Sure enough, even though everything (according to the Southwest ticket agents) was done correctly, the ticket agents ended up calling Southwest's Dallas headquarters to get a special dispensation to release my tickets.   Everyone at Southwest was very helpful; at one point a ticket agent was holding a phone in each ear at the same time.  The whole process took about 30 minutes to issue a boarding pass.  Southwest has an open seating policy, so I was told to talk to the gate agents when boarding.  


Going through security was uneventful.  The case fit through the xray machine with a little room to spare.  I was not asked to remove the end pin.  Walking through the airport was kind of fun; people smiled and parents would tell their children to look at the cello.  I might as well have been leading a llama through the concourse, because as everyone knows people love llamas.


At the gate, I asked the boarding agent about when to board and what were Southwest's regulations concerning the transport of cellos.  She didn't know if there were any regulations, but she let me board with the families with small children so I could find a pair of seats more easily.  Once on board, the stewards and stewardesses knew that I was flying with a cello and led me to the last row of seats. I also asked for a seat belt extender to secure the cello case.  The space between the seats was not large enough to fit the cello case so I had to pull up the seat cushion.  


I've always been hesitant about pulling up seat cushions on a plane ever since my cell phone slipped between the seat crack on a flight soon after 9/11.  When I pulled up the seat cushion that time, I found a number of bullets in the seat presumably left by an air marshall.  Long story short, the flight was delayed and I received the evil eye from the other passengers. 

Happily there were only stale peanuts and I secured the cello by sliding the seatbelt through the handle.  The flight was uneventful and the cello arrived safe and sound.


I would say that the important stuff about transporting a cello on a plane are:

1. Give yourself lots of time to deal with ticketing problems and to avoid the stress of trying to make a flight when there are long security lines.
2. Talk to the ticket agents and gate agents, they will help expedite the process.
3. Take a direct flight if you can.
4. If you have to take connecting flight, give yourself plenty of time between connections.
5. Smile a lot, it really does help make things go smoothly.

Flying with a cello, part 1

I moved to Texas in June but I left my cello in Maryland because I thought I would be spending more time in Maryland.  However, it didn't work out that way so I'm flying it back with me.  I researched a couple of options, one of which was to purchase a a flight case for my Bobelock 2000 cello case with the intent of checking the cello in as luggage.  Apparently there are travel cases for Bobelock cases, but they are not the same as flight cases.  The only flight case I could find was for Bam cases and they cost around $700.  Not sure that my Bobelock case would fit  and that the return flight was less than $100, I decided to buy my cello a seat.


I usually fly Southwest between San Antonio and Baltimore (BWI) because they have a direct and they're convenient.  I first booked my round trip ticket online and called reservations to book a seat for the cello.  The agent was very helpful and she reserved a seat for the cello and tied that ticket to my roundtrip ticket.  When buying a seat for an inanimate object, they issue the ticket in your name with an IXS suffix (Inanimate Xtra Seat, I think).  It took more than one try and they issued a couple of confirmation emails but it all seemed to work - until I tried to check.


When I tried to checkin at the kiosk, I was told to see a gate agent.  Apparently, Southwest's reservation system was unhappy that my return trip included a second seat under my name and it wouldn't let me checkin.  Southwest solution was to break my reservations into individual legs, i.e. three one way tickets.  They issued new confirmation tickets for the return flight, but the cello's ticket lacks the IXS suffix, so we shall see.

Augmented Reality and Android: This is not the droid I'm looking for

Brother Cavil: In all your travels, have you ever seen a star go supernova?

Ellen Tigh: No.

Brother Cavil: No? Well, I have. I saw a star explode and send out the building blocks of the Universe. Other stars, other planets and eventually other life. A supernova! Creation itself! I was there. I wanted to see it and be part of the moment. And you know how I perceived one of the most glorious events in the universe? With these ridiculous gelatinous orbs in my skull! With eyes designed to perceive only a tiny fraction of the EM spectrum. With ears designed only to hear vibrations in the air.

Battlestar Galactica [No Exit]

My Blackberry died of a broken USB connector a few weeks back and I replaced it with a myTouch; or as it is more geekily known, the second Android phone (this distinction in branding is important and I'll get back to it later). As with any new toy/technology, I can't resist loading the latest geek ware in an effort to break it. August was the Augmented Reality (AR) news month with Bruce Sterling playing beaming father to the fledgling industry; so what better way to try out new hardware than loading up a bunch of apps that make full use of the platform?

The Android based phones seems to be platform du jour for AR because of hardware access to the GPS, accelerometer, compass, and 3G (for the majority of the time). Augmented reality apps for iPhones have been recently released with the advent of the iPhone 3gs, and since the iPhone apps market is estimated to be worth 2.4 billion USD, in comparison to Android's miniscule apps market, expect AR apps to migrate to the iPhone real soon now. Returning to the branding issue, Joe Lamantia raises the geek-to-chic issue that will impact the future development of AR platforms.

I loaded Layar, Wikitude, and Gamaray and used them for a couple of weeks across different environments ranging from the suburbs and urban areas in Texas, Maryland, and New York City, inside and outside buildings, and while stationary and moving (car, train, plane). Comparisons are not terribly useful since they are very similar in function. In general, Layar currently has the most data layers but searching seems limited; for example a search for "sushi" returned no results but "food" did return results, but not always relevant. Wikitude imprints the closest street address if you take a photo, but I don't know if it writes the location data to the exif. Gamaray allows you to create content (including the insertion of 3D objects) without a developer key à la KML.

There are annoyances related to hardware. Getting a position fix on the myTouch's GPS can take over 2 minutes, and sometimes a reboot is necessary to kick the GPS out of its locational funk. Apparently, the iPhone also suffers from this problem. Layar and Wikitude handle this by using the last known position and Gamaray just refuses to start with a current position fix. AR apps are frequently rendered useless when in inside a building or in a moving vehicle. New York City's buildings create an urban canyon effect that hinders reception of GPS signals. This is unfortunate because AR apps seem to be targeted at the built environment.

For me, the biggest shortcomings of the current crop of AR apps has been the user experience, the lack interesting content, and the use of only locational sensors. I'll address them individually within the context of Joe Lamantia's article detailing four common AR user interaction patterns:

• Heads Up Display - "information about the real objects are added to a fixed point of view, typically the focus of the user’s visual field."
• Tricorder - "add pieces of information to an existing real-world experience, representing them directly within the combined, augmented-reality, or mixed-reality experience."
• Holochess - "adds new and wholly virtual objects directly into the augmented experience, combining them with existing, real objects. The virtual items in Holochess interaction patterns often interact with one another—and sometimes with the real elements of the mixed-reality experience."
• X-ray vision - "simulates seeing beneath the surface of objects, people, or places, showing their internal structure or contents. AR experiences using the X-ray Vision pattern often use a combination of projection and rendering—frequently, a schematic or abstracted rendering—of the object of interest"

UX sucks

Layar, Wikitude and Gamaray use a hybrid of the the Heads Up Display and the the Tricorder. My main pet peeve is the use of the Heads Up Display interaction pattern which is represented as a radar like scope with a wedge representing the field of view. Don't get me wrong, I spent many happy hours playing Battle Zone, but when I have to physically rotate the device (and myself) to see objects to my side or behind me, a simple 2D map is a more practical interface.

The Tricorder pattern is equally annoying, since this involves holding up the phone in front of you while scanning the horizon for alien life forms. Doing this made me feel like an alien in Herald Square. While I don't have a problem being an alien, it also made me feel like one of those self-important dorks who wore a blue-tooth headset constantly when they first came out.

Oh the promise!

Gamaray differs from Layar and Wikitude in that it implements the Holochess pattern by allowing users to place 3D object in the field view. This where things start getting interesting because users can interact with the application and reality instead of passively viewing info bubbles. It's not a far jump to the locative art installations as described in William Gibson's book Spook Country. The Mannahatta Project reconstructs the changing ecology of Manhattan back to 1609, and a AR app could overlay the historical view over the current view. AR applications are a natural fit for these visualizations. Temporal overlays could also be used to overlay houses,tax parcels, and demographic data in areas hit by tornados or hurricanes for use by emergency workers and insurance adjusters.

Where's the content?

Granted that AR is in its infancy, but the currently available content detracts from the idea of AR as a useful tool. A number of apps use Wikipedia as a starting point for content, but the entries in Wikipedia are scale inappropriate and are spatially coarse to be of interest. Tobler's law and its corollaries lands with a heavy thunk. Wikitude, Layar, and Gamaray all support user created content but there doesn't appear to be standardization around any particular content model. Crowd sourcing geospatial content worked well for Google, but Google had the advantage of a single data schema in KML as well as a single API.

AR is a natural fit for built environments but geospatial data is rarely collected at the resolution of individual manmade structures and data collection at this scale often means 3D. CAD data does exist for many urban areas but it is frequently locked up in hard drives of various planning agencies and engineering companies; its difficult to imagine that any of that data will be readily available. An alternative is Sketchup and apps are starting emerge, but none on a mobile platform so far.

Not significantly advanced technology

Considering that 18 years ago I carried a GPS that was larger and heavier than my current notebook and 7 years ago my Garmin Etrex was bigger than my circa 2000 Motorola Startac cell phone, you would think that I would be pretty happy with a device that combines GPS, compass, cell phone, camera, internet terminal, and music player into a single package. Actually, I am happy that I own this technological wonder that can begin to implement AR, but where AR apps fail to be indistinguishable from magic is that they don't use all the sensors. Amidst all the trumpeting that the compass was the killer feature on Android, features such as bluetooth, wifi, and of course the camera seemed to have been forgotten.

Makers have been busily hacking at phone cameras to make scanners and there are accessories to turn the phone camera into a microscope. An application that makes use of image recognition to identify objects in the field of view instead of using proximity and bearing to the phone's location would be infinitely more magical the current applications. An AR app can conceivably take a picture and send it to an image recognition service and then provide additional information based on the picture just like the folks at astronomy.net. That would be magic indeed.

Better than 11

Mono, stereo, ... spatial!

FOSS4G Paper Accepted

It's been a while since the last update; I've been busy with a move back to the Lone Star State. I figure any place with 2 for a dollar tacos is a good place to be when it comes to weathering the current economic situation.

FOSS4G 2009 announced its selection for paper presentations and my abstract was selected. Titled, "Getting a Pulse", its an attempt to get a census of active users of open source geospatial software. It was partly inspired by an earlier post about Neo vs Paleo geographers and a continuation ideas I presented at the MUM/EOGEO 2005 Conference titled Open Source vs Closed Source: 42 Midgets and a Lion. The tongue-in-cheek and politically incorrect title was from a popular 2005 internet meme based on a fake BBC news report and was meant frame the discussion in terms of the likelihood that any of the numerous open source geospatial projects could replace the major geospatial software vendors. Midgets and lions aside, I think that the geospatial industry is in a state of flux and that there are new contenders entering the market every day. The very nature of the geospatial market is changing dramatically.

For this presentation, I'm extracting the info from the user mailing lists and inserting them into PostgreSQL for giggles and because SQL is more fun for analysis than java. If I can figure a way to find the location from generic email addresses such as gmail, I'll add geometry to each record. So far I've written a module to extract information from mailman list archives, and extracting data from sourceforge list archives is next. I'll publish all the data and tools when I'm done.

CrisisCamp

CrisisCamp: June 12-14, 2009

Ignite

On Friday night, Ignite was held at the World Bank. The talks were excellent and although I didn't take notes, people twittered where to find their presentations.

@ajturner noted that it was "perhaps the swankiest Ignite ever, hosted at the World Bank." Credit for the photo also goes to @ajturner . Also, the World Bank asked that we drink all the booze.

CrisisCamp Day 1

Crisis Mapping in Sudan

Led by Patrick Meier via Skype from Khartoum, the discussion was about a UNDP project, Threat and Risk Mapping Analysis (TRMA). TMRA maps microlevel problems and threat and risk indicators to provide better information when implementing development programs and to avoid causing or exacerbating problems. The data is collected by focus groups and capture the rich local knowledge through capturing information on printed maps. The data is entered into a GIS and so far has collected 6000+ data points and mapped over 700 new villages. Patrick also talked about 4W (who, what, when, where) crisis mapping tool based on open source. 4W shows emerging trends, situational pressures, market routes and critical fault lines through out a region. On the interoperability front, the project has been part of establishing an information sharing group and a data sharing protocol among UN agencies for baseline data. Data sharing has been problematic and has led to duplication of effort, the desire not to share data was describe as "DHD, data hugging disorder." Mesh4X was also discussed a means for data sharing across multiple platforms and high latency networks (read disconnected clients).

The discussion turned to data collection, especially using SMS and occasionally MMS. Andrew Turner mentioned the Youth Assets, where SMS is used by children to perform emotional mapping. It was generally agreed that SMS is the lowest common denominator in terms of a protocol and platform for data collection. However, SMS was not always ideal and that there was some difficulty in getting structured data in Ushahidi. FrontlineSMS was also mentioned as a means for coordinating receipt and delivery of SMS messages when Internet access is not available.

The last part of the discussion tied the technology back with the practice of data collection in the field. IFAD, the International Fund for Agricultural Development published Good Practices in Participatory Mapping, which is a practical guide for data collection in the field.

UX, Usability, Visualization

This session was supposed to be about user experience, usability, and visualization, but it focused more on reliability of crowd sourced data and verification of the data to provide reliable and actionable information. Swift, a framework for verifying crowdsourced information, was discussed.

Part of this discussion was dissemination mechanisms, and delivery crisis information on the most common platforms, which are radio and tv.

Again, SMS was noted as being one of the last communication systems to go down during a communication surge situation that frequently occurs during crisis. The question was asked, "How do you text a 911 call?" if that is the case. The point was made that SMS should not be used as a push mechanism for delivery alerts; rather, SMS should be used for listening to requests and this is a more efficient use of the technology.

There was also a discussion of authoritative or official sources and their role in a crisis. There are 2 national systems to alert during emergency IPAWS and CMAS for delivering alerts.

Crowd Sourcing Situational Awareness

This session was led by David Stephenson. (Unfortunately, I walked in late to this session, missing about half the session).

"How do you get institutional buy-in form the government?" was the question posed during the session. Buy-in maybe defacto because it can not keep up in comparison to crowdsourcing. Stephenson noted that David Robinson of Princeton's Center for Information Technology Policy advocates that the government should publish data and leave the interpretations to the consumers of the data. It was noted that data.gov will go from 80 feeds to 100,000 feeds in a month.

With the imminent flood of data, data analysis and visualization tools such as swivel and manyeyes were mentioned.

Swine Flu Response

Andrew Wilson of the Department of Health and Human service led a discussion on his work using social media in response to the Swine Flu pandemic. In addition to the official channel, pandemicflu.gov, he employed FaceBook, twiiter and podcasts to provide and collect information about H1N1. One if his strategies was to use tweeters with large numbers of followers to retweet information to create an amplifying effect. One area that they did relatively poorly was mapping and he pointed to the Ushahidi Swine Flu map as a better implementation.

Lazy Web Disaster

At WhereCamp2009, Mikel Maron led a sesssion where participants could yell out ideas or projects that they would like to see but never implement. The Lazy Web Disaster session collected these ideas on twitter - the results.

CrisisCamp Day 2

Recap of Day 1

The second day started with a long recap of the session from the first day.

It was evident in the previous sessions that there was a tension between domestic and international responses to crisis. Despite this tension there are commonalities and possibly low hanging fruit between the two. Lack of a common a common vocabulary was identified as one of the obstacles that prevent organizations from working together. For example, there are 60,000 organizations nationally that provide crisis management and relief services, but they don't share a common vocabulary. As noted in the Crisis Mapping in the Sudan session the UN has developed a wiki to build a common terminology.

Noel Dickover asked, "What is the coalescing function? Was it crisis response? Why are you here?"

Greg Elin responded that crisis cuts across multiple communities, opens doors and gets people at the table. Crisis activates the bureaucracy, creates opportunities, and opens doors. Bureaucracy may not move or change, but there is still a desire to tap into opportunities, some of which are generated by social media.

The discussion turned to preparedness. Succesful crisis management is based on agility, reacting to the unexpected. So, what are the tools available to handle the situation? Furthermore, is there way to incentivize being prepared. Ready.gov is an official channel for disaster preparedness information, but why don't people know about it and if they did would they care?

It was note that ready.gov is hamstrung because its in the government space, and that being a government organization hampers its effectives for disseminating the message. Preparedness will require a cultural change, and that current fear based communications do not work.

The discussion turned to what could the participants do as a result of the CrisisCamp; how can these dialogue and ideas become actionable. Concrete steps to continue the work of CrisisCamp were:

1. develop a common language
2. concentrate on what first by identifying the gaps and not jump to how, i.e. technology
3. engage more people to provide the force multiplier, grow the 80 participants to a network of 8000 people
4. provide tools that are simple to use. Google tools don't require training, so that is the bar to shoot for
5. create interoperable applications as demonstation
6. create a demo scenario and build apps around the scenario
7. creation of a wiki: crisiscommons.org
8. creation of crisiscampdc.ning.com - common community platform
9. create a common design document/template for crisis apps
10. crisiscamp messaging matrix

The remaining sessions were devoted to discussion on implementation of these ideas.

Finally, a summary of CrisisCamp culled from twitter is available.