Understanding Augmented Reality

Fabrizio1

With the launch of apps like Sphero’s The Rolling Dead and technology like Google Glass forging the way, it is a very exciting time for augmented reality. Of course, similar claims have been made about virtual and augmented reality since the 80’s. Every year, someone predicts: this is the year our holodeck dreams will finally be realized. The full holodeck experience is still a long way off, but the explosion of powerful, sensor-equipped mobile devices ensures the ubiquity of augmented reality is just around the corner. The surest evidence of this fact is the increasing availability of SDKs (like the Sphero AR SDK/Unity Plugin) allowing developers to create exciting, immersive, augmented reality experiences without any special knowledge of computer vision, sensor fusion, or any of the other specialized technologies behind modern AR engines.

Why Now?

There is only one answer: the hardware. Twenty years ago, essentially every component of a mobile AR platform was prohibitively large, expensive, or technologically insufficient. Processing video in real-time requires an extraordinary amount of computational power. This brings back memories of Steve Feiner’s “Touring Machine”, which required the user to wear a backpack computer! As for digital cameras, accelerometers, gyroscopes and mobile displays, prices were high and quality was low. The smartphone revolution changed all that. Chances are, you have a capable augmented reality device in your pocket.

What is AR… Really?

At a recent augmented reality conference, I was showing an elderly woman the Sphero app “Sharky the Beaver”. The app allows you to control Sphero (a robotic ball) from a smart phone or tablet. The display shows video coming from the back facing camera of the mobile device with one significant augmentation: an adorably plump, animated beaver rendered over the ball. If you are looking at the screen, it appears as if you are driving a 3D cartoon character around your living room.

She pointed at the screen and said “I see the beaver here…but I don’t see it here,” pointing at the real ball sitting on the floor. “How is that augmented reality?” To be sure, I had a good laugh on the inside. But, the truth is, she makes a great point. Augmented reality is an audacious term, undeserved almost everywhere it is applied. No holodecks quite yet, unfortunately.

To avoid this confusion, I try to use more specific language. Apps like 13th Labs/Mojang’s “Minecraft Reality” should probably be called augmented video. That is what they literally do: render virtual content into a video feed so as to deceive the user into thinking it is real. Location based games are often categorized as augmented reality too. Google’s “Ingress”, for instance, might be more appropriately called augmented cartography. While Google Glass has yet to arrive commercially, its small, off-center display makes it unsuitable for immersive experiences. Likely, Google Glass applications will be appropriately described as annotated reality.

You may chuckle and think I am taking this analysis too far. But soon, real and virtual sensory information will be even more mixed up than they are now. Phrases like augmented reality will lose their edge and it will be important to distinguish exactly what is being augmented. If you want a real chuckle, try imagining if other senses were augmented.

Robotics and the Sphero AR SDK

How about “Sharky the Beaver”? One could make a compelling case that this is an example of augmented video. Rendering a virtual character into a video stream certainly qualifies, but I think there is more to it than that.

Sphero is a robot. It can move. It can behave autonomously. It is a digital device with a mechanical toehold in reality. We may not have force-field treadmills quite yet, but we can augment one tiny round piece of reality. The Sphero AR Unity Plugin allows developers with a little .NET experience and no background in robotics to coordinate robot behavior with virtual content.

Practically speaking, what does this mean? In terms of gameplay, it means quite a lot. Have you ever wrecked a toy truck into a pretend building? Have you ever made “countries” by dividing up a room with scotch tape? Have you ever uttered the phrase “the carpet is lava!”? If your childhood was as fun as mine you are going to love what augmented reality has to offer in the next ten years. If playing pretend is an alien concept, don’t worry. We’ll bring you up to speed.

To see Sphero’s augmented reality capabilities in action, check out our new app The Rolling Dead, now available on iOS and Android. You can learn more about AR with Sphero here.

How to Load Apps on Google Glass [VIDEO]

Google_Glass_Explorer_Edition.jpeg
Google Glass is a new paradigm in the world of connected computers, and the Orbotix team was lucky enough to receive this one last week at Google I/O in San Francisco. We’ve spent the majority of that week playing with our Glass, and have since loaded some apps of our own. Here’s how we made it happen it using Google Sky as our example.

For this tutorial we will show you how install Google Sky (also known as stardroid) without rooting your Glass or swapping roms. This is the same basic procedure you would go through to manually load an app on an Android phone, but since Google Glass has a limited amount of input, the specific app you install matters. Google Sky is perfect! It doesn’t require any user input other than the accelerometer and gyroscopic sensor data, but it will show you an amazing amount of information when looking at the night sky.

Before we begin it is important that you already have the Android SDK, most importantly you need ADB (android debug bridge). You also need to be somewhat familiar with opening a new terminal window. We used Mac 10.8+, but these same basic directions will work for Linux and might need a little modification for Windows. We also assume you download files to your home Download directory, but you can change this as needed. (~/Downloads)

Download and Install the Android SDK (http://developer.android.com/sdk/index.html). We find it helpful to have adb in your PATH so you can call it from anywhere (you may want to add it to your .bashrc or whatever you are using).

First things first, Put your Glass in “Debug Mode” in the settings on the Glass itself. Locate the device info and scroll right to turn Debug Mode on. Then plug your Glass into your computer using the USB micro cable.

Now grab a version of Google Sky that you want to install (this is the same for any Android .apk file).

Simply load the .apk file onto the Glass with adb.

After the .apk is loaded onto Glass you now need to use adb shell to run it. The trick here is that you need to figure out what Activity to run. You can use the adb command dumpsys to discover a plethora of information about your Glass. You can figure out the main activity for the .apk with a little help from grep, and since it was the last thing loaded we can also narrow this information down with a tail command.

This might take some time depending on your system, but it will eventually return something similar to this:

intent={act=android.intent.action.MAIN flg=0×10000000 cmp=com.google.android.stardroid/.activities.SplashScreenActivity}
android.intent.action.MAIN:

You now have the right information to start the app on Glass. We use the am command to start the app on Glass. Simply reformat the Activity call to follow the same format as the stardroid example call below:

At this point tap your Glass on and the app should be running. You will need to “down swipe” on your glass to cancel the agreement window. If you see the app running, disconnect, go outside, and check this out at nighttime! Warning: you may feel like a terminator with some kind of super power.

Note: this is not part of the Official Glass Mirror API that is displayed on your timeline and running apps full time like this is a drain on the battery, which is not in Google’s application design guidelines. It merely demonstrates how to sideload an app on Glass in your terminal without voiding your warranty.

Google_Glass_logo
Why we think this is awesome:

Some people will say that you can do this exact same thing with your smartphone, and we agree you can. But what fascinates us about this is a constant stream of information about our world, in real time, with little or no effort on our part. We can imagine apps like Google Googles, and other augmented reality apps and games becoming extremely valid in a world where they were just neat to show off on a phone. Nobody holds their phone up looking through the camera as they walk around – it’s just hard to do.

This all excites us when we relate it back to what we are doing at Orbotix. Augmented reality is part of our DNA, as Sphero is the perfect moving, trackable robot that allows nearly any 3D digital experience to be seamlessly stitched into our real world with little effort from the user. You see a 3D digital character like Sharky the Beaver magically appear on your living room floor and you can control it around your house.

We hope for this same magical experience from Google Glass as it is fully developed. It should naturally pop up information about the things around us based on our Google Now history and what I am looking at. If I have to manually tell it to “scan” a bar code, it’s doing it wrong. For now, this is the closest we have ever felt to being in a science fiction movie. Google Glass is an incredible step toward an amazing future and we’re looking forward to being a part of it.

Google I/O 2013

209399_10152794247720109_1006182063_o

Google I/O is one of the conferences the Sphero team looks forward to attending and exhibiting every year. This year was no different. Monday kicked things off with us showing Sphero at the Google campus in Mountain View with Google [X] employees. Tuesday night we had an interactive photo-booth where attendees could take picture with the Android character. We got over 300 pictures!

A notable highlight from the trip was giving the co-founder of Android, Rich Miner (pictured above) his very own custom Sphero. We’re big fans of what he helped create. Also Steve Aoki had something to say about Sphero.

We had a great time. See you next year, Google!

Sphero Augmented Reality Technology Now Available for Developers

UnityBlog

Sphero is at the forefront of some game changing technology with augmented reality. And now, it’s getting even better. Today we’re excited to announce a new Sphero Augmented Reality SDK available on Github, as well as a new Sphero Augmented Reality plug-in for Unity. What does this mean for you? More games for Sphero, and more fun.

Now developers can simply insert their 3D models (think Sharky the Beaver) and create games around them – all using Sphero as the moving fiducial. The plug-in does all the vision detection by automatically coordinating the camera and ball position, distance, orientation and more. The plug-in will form a bridge between Unity and Sphero’s native Android and iOS SDK’s, and voilà. Designed for iOS, with an update that will include Android coming soon, this is the first Unity plug-in that allows developers to create augmented reality apps for a physical device. Once the Android update is ready, it also means that we will be able to release Sphero augmented reality apps on iOS and Android at the same time.

We’ve also launched the Sphero App Bounty program. Developers can win some awesome prizes for creating the apps we’ve always wanted to build, but haven’t had the time to create. Check out the list and stay posted as it continues to grow! In the meantime, you can send any questions, concerns, or pick-up-lines directly to our developers on our forums.

Windows Phone Love: Unofficial Sphero App Now Available

windowsphoneSphero operates on two main platforms – iOS and Android. But because it’s a programmable robot, Sphero can be hacked to work with just about any device. Using our open API, third party developers have controlled Sphero with everything from AR Drones to Xbox controllers. And now thanks to one of these outside devs, Sphero has an app for Windows Phone.

The app is similar to our main Sphero app, in that it allows you to drive Sphero, change colors, and take photos. The only difference is that now Windows Phone users can take Sphero for a spin! Check the developer’s blog for more information and download today. In case you missed the news, we’ve also recently launched App Bounty where devs can check out game ideas for Sphero and get rewarded for their efforts. Stay posted for more releases!