Augmented reality and the Powerhouse images in the Commons (or interesting things clever people do with your data #7215)

On Saturday night at our (very rainy) Common Ground meetup in Sydney, Rob Manson and Alex Young from BuildAR demonstrated the first version of their augmented reality mobile toolkit using images from the Powerhouse’s geocoded photographs in the Commons on Flickr.

This work riffs around the early mashup from Paul Hagon where he combined the historic photos with Google’s Street View; and the ABC’s Sydney Sidetracks project.

But then makes it mobile – replacing the Street View with the actual view through the camera of your mobile phone.

I asked Rob a few questions –

F&N – What is this Augmented Reality thing you’ve built? What does it do?

The first service is BuildAR and it is a service built upon the Mobile Reality Browser called Layar.

Layar uses the GPS on your mobile to work out where in the world you are, then it uses the digital compass to work out which direction you’re facing (e.g. your orientation). From this it can build a model of the objects and places around you. Then as you hold up your mobile and pan around, it can overlay information on the live video from your camera that you see to highlight where these objects and places are.

BuildAR let’s you quickly and easily add, search and manage your own
collection of Points of Interest (POIs) to create your own Augmented Reality layer. You can do this via a standard PC web browser, or you can do it via your mobile phone. You can create a free personal account and get started straight away creating your own private POIs or you can make public POIs that other people can view too. All it takes is a few clicks and they are shared or published in real-time.

You can also use the service to create fully branded and customised layers.

We’re constantly releasing new features including groups so you can share private POIs with others, rich graphs so you can view when and how people are using your POIs and custom mobile websites that each of the POIs can link to. We can even customise layers to make them really interactive so the POIs you see are based on where you’ve been, other POIs you’ve interacted with, the time of day or any range of options. Treasure hunts are a great example of this.

How did you use the Powerhouse data?

We’re in the process of creating layers for a lot of people at the moment and another great example is with the Powerhouse images that were released into the Flickr Commons. We loaded over 400 of these images as public POIs so now you can wander around Sydney with your phone and see beautiful historic images of the local area around you. You can then just tap on the POI/photo and you get the option to go directly to the Flickr page for that image, or even better straight to the Powerhouse page with all the historic information and the original image.

I spent the afternoon with my son the other day wandering around looking at images of our local area. Neither of us knew that Bondi/Tamarama used to have an Aquarium and it has opened up a whole new world for us to explore.

How easy was it to use Layar? What are the benefits?

It was reasonably straight forward, but it was a very technical process.

That’s largely why we created BuildAR – so other people can create and manage their own POIs by just pointing and clicking, or wandering around and using their mobile.

The benefits are that it is a great system with quite an open API. They’re gaining a lot of traction and I think the “browser with layers” approach is much better than creating dedicated applications.

This is much more along the lines of how the web works.

If you want to create something then you just create a website that uses standards based HTML/CSS. It just wouldn’t make sense for you to also have to create your own browser too. That’s the old model from before the 90’s and we’ve all learned a lot and come a long way since then.

Layar are releasing some great new features soon too, like supporting 3D models and animations and support for more mobile device types. They can focus on that and we can just focus on creating great layers and tools that make it easy to create and manage layers.

What data sets were you looking to use? How easy was it to use etc?

We’re looking for either content that’s compelling or data that’s useful. The Powerhouse images are a great example of compelling information and the team at the Powerhouse made it really easy to integrate into our application (thanks Luke and Paula!).

Very soon we’ll be releasing an option that lets you upload a batch file of POIs or just point it to a GeoRSS feed and you’ll be done. Couldn’t get much easier than that!

Another great example of compelling content we’re currently working on is with Sculpture by the Sea. This is a beautiful outdoor experience and is a perfect fit for mobile Augmented Reality.

We’re also doing quite a bit of work in the Government 2.0 and Open Data movement and we’re currently working on a range of layers that utilise the really useful public data that’s being released. Our goal is to help this data become more “situated” and therefore hopefully more relevant . . . then on top of that we’re opening up layers of social interaction to add even more value.

This is a really interesting time with a lot of social change on the horizon. The combination of Augmented Reality and Open Data is something that is literally changing the way we see our world.

What platforms does it run on? Will it be easy to port to the iPhone?

At the backend BuildAR is simply a relatively open API and we implemented that all on our Linux based servers. On the Layar browser side it currently runs on Android based devices and will be released on the iPhone 3GS and some other platforms soon too. The Layar team are working hard to port and enhance this whole application and the goal is to support any phone that has GPS and a digital compass built-in.

I think in the near term future you’ll see GPS and digital compasses start to spread back onto netbooks and laptops and then the tablet computers that will be released soon.

You were demo-ing another AR application at the Web Week launch party. Tell me about it?

This was a “marker” based AR project, an ARt exhibition collaboration with Yiying Lu who created the “Fail Whale” for twitter. Basically you just hold up an illustration created by Yiying, on a postcard or a t-shirt, in front of a camera connected to an internet connected computer. The application we created then recognises the image and then projects a simple Fail Whale animation over the top of the marker.

This also loads that last 30 tweets with the #wds09 hashtag and randomly displays one of them every 45 seconds. It’s all kinda self-referential and tongue-in-cheek and is a great way to play with and interact with Yiying’s beautiful illustrations.

You can try this on your own computer too. All you need is an internet connected computer, Flash installed on your browser and a working webcam. Just visit the project website and have a play or just watch the video to see how it works.

It is still quite early days with this technology and the light levels can really impact how well it works, but AR is definitely something that has an impact when you experience it.

“You are what you tweet” Augmented Reality exhibition from Rob Manson on Vimeo.

We are obviously in the early days of mobile phone AR. How do you see it developing?

Well, I’m working on a broader research project on Pervasive Computing and I think this is a core part of that evolution. The interfaces are still quite clunky and having to hold up and wave around your phone is still quite a clumsy experience.

I think quite soon we’ll see more immersive display devices start to spread. I’m running a session on this at Web Directions South and we use this underlying theory to inform most of our business/product strategy development.

Basically the distance between the network and the user is collapsing. The distance between the display and the user is collapsing. And the distance between the physical interface (e.g. think of gestures) and the user is also shrinking. This means our overall experience of space and even who we are is changing.

This all seems a bit futuristic, but glasses with displays built-into them should start to spread quite soon, all powered by mobile devices. And there’ll be even more interesting options too. Just think how quickly iPhones and Bluetooth headsets have become common everyday objects.

The opposite side of this is the spread of wireless digital cameras.

Combine the two and you open the door to rich and immersive Augmented Reality where you can shift your perspective constantly and freely.

I think this is the start of something really fascinating!