Understanding Augmented Reality


With the launch of apps like Sphero’s The Rolling Dead and technology like Google Glass forging the way, it is a very exciting time for augmented reality. Of course, similar claims have been made about virtual and augmented reality since the 80’s. Every year, someone predicts: this is the year our holodeck dreams will finally be realized. The full holodeck experience is still a long way off, but the explosion of powerful, sensor-equipped mobile devices ensures the ubiquity of augmented reality is just around the corner. The surest evidence of this fact is the increasing availability of SDKs (like the Sphero AR SDK/Unity Plugin) allowing developers to create exciting, immersive, augmented reality experiences without any special knowledge of computer vision, sensor fusion, or any of the other specialized technologies behind modern AR engines.

Why Now?

There is only one answer: the hardware. Twenty years ago, essentially every component of a mobile AR platform was prohibitively large, expensive, or technologically insufficient. Processing video in real-time requires an extraordinary amount of computational power. This brings back memories of Steve Feiner’s “Touring Machine”, which required the user to wear a backpack computer! As for digital cameras, accelerometers, gyroscopes and mobile displays, prices were high and quality was low. The smartphone revolution changed all that. Chances are, you have a capable augmented reality device in your pocket.

What is AR”¦ Really?

At a recent augmented reality conference, I was showing an elderly woman the Sphero app “Sharky the Beaver”. The app allows you to control Sphero (a robotic ball) from a smart phone or tablet. The display shows video coming from the back facing camera of the mobile device with one significant augmentation: an adorably plump, animated beaver rendered over the ball. If you are looking at the screen, it appears as if you are driving a 3D cartoon character around your living room.

She pointed at the screen and said “I see the beaver here”¦but I don’t see it here,” pointing at the real ball sitting on the floor. “How is that augmented reality?” To be sure, I had a good laugh on the inside. But, the truth is, she makes a great point. Augmented reality is an audacious term, undeserved almost everywhere it is applied. No holodecks quite yet, unfortunately.

To avoid this confusion, I try to use more specific language. Apps like 13th Labs/Mojang’s “Minecraft Reality” should probably be called augmented video. That is what they literally do: render virtual content into a video feed so as to deceive the user into thinking it is real. Location based games are often categorized as augmented reality too. Google’s “Ingress”, for instance, might be more appropriately called augmented cartography. While Google Glass has yet to arrive commercially, its small, off-center display makes it unsuitable for immersive experiences. Likely, Google Glass applications will be appropriately described as annotated reality.

You may chuckle and think I am taking this analysis too far. But soon, real and virtual sensory information will be even more mixed up than they are now. Phrases like augmented reality will lose their edge and it will be important to distinguish exactly what is being augmented. If you want a real chuckle, try imagining if other senses were augmented.

Robotics and the Sphero AR SDK

How about “Sharky the Beaver”? One could make a compelling case that this is an example of augmented video. Rendering a virtual character into a video stream certainly qualifies, but I think there is more to it than that.

Sphero is a robot. It can move. It can behave autonomously. It is a digital device with a mechanical toehold in reality. We may not have force-field treadmills quite yet, but we can augment one tiny round piece of reality. The Sphero AR Unity Plugin allows developers with a little .NET experience and no background in robotics to coordinate robot behavior with virtual content.

Practically speaking, what does this mean? In terms of gameplay, it means quite a lot. Have you ever wrecked a toy truck into a pretend building? Have you ever made “countries” by dividing up a room with scotch tape? Have you ever uttered the phrase “the carpet is lava!”? If your childhood was as fun as mine you are going to love what augmented reality has to offer in the next ten years. If playing pretend is an alien concept, don’t worry. We’ll bring you up to speed.

To see Sphero’s augmented reality capabilities in action, check out our new app The Rolling Dead, now available on iOS and Android. You can learn more about AR with Sphero here.