RCOS Fall 2013 Proposal

Overview: MirageMobile is a mobile sdk for building large scale augmented reality application. It includes the ability to use markless ar db to show opengl shifted to a particular homography. it will tie into out backend service MirageWebApp which allows you to store hundreds of thousands of markless AR plus the 3D that goes along with it.

This project will compete with other up and coming 3D frameworks like wikitude and vuforia for the AR space.

So far we have debuggable AR and 3D working on the product. This semester we want to make the debugging for the AR code easier by adding testing framework for changes in our AR code. We are building this on devise instead of on a desktop application in order to see the full performance the software will have on the device so far. I am working on getting the application to first be testable by rendering opengl on a image that includes a markless AR. Then we will let the test environment have the ability to include looping video for showing markless AR. This will be a more stable environment for testing out future iteration of our matching algorithms.

Development schedule:

Weeks 0-4: Working on static images to test out the matching as well as adding code for debugging matching on static images

Weeks 5-7 : Working on getting the code to work on a pattern database that includes features and keypoints of potential markless targets.

Weeks 11-13: Working on getting the opengl to work on the static images target

Week 14-16 (November 18 - End  : Finding a way to get video to show instead of static images.

One Engine For 2 Products

One of the goals of mirage engine (new name) is for it to work on multiple systems at once. So we made the library now work for both computers and android mobile phones. Without change the library now it can run matching algorithms for both. In order to do this we used Binary Descriptors. Binary Descriptors used much less data and worked much faster than the locality based descriptors. Which allowed us to use it on a phone, but give us the speed need to run it on a large set of servers.

Right now our engine runs both our android mobile application and our rails app at trymirage.com. Hopefully we can find a way to extend it to other systems so that more people can more easily make image recognition applications.

Update to the Server

So we changed the backend for the server. We went from a playframework to a combination of a java application and rails combined framework. The application uses rails to call a matching algorithm for each individual users database. It will be operational soon at developer.trymirage.com.

Figuring out the camera calibration.

So, right now we have been bogged down by work from other things. I am working on getting things cleared away, but for the next few weeks little development will be done. We have made a break through figuring out the homography. When I do start up working fulltime again I will be working on getting this new sound homography to make it into a modelview matrix.

Update on Recognition Server

So I wanted to give a update on our progress.

• Using homography checking and adding a system to score the matches I narrowed down our matching from 5 possible guesses to 1. 
• We also sped up the matching algorithm to around 2-3 seconds per match for our binary descriptors. 
• We also changed from a json system to a proto file based system that is smaller and faster. This is proto buffer by google which is used in the maps project as well as many others.
• We also created one load function so now all the files are loaded once instead of for every matching process. All these clusters are stored into ram.

We are now working on getting the mobile application to send these to the server code. We have a prototype up and running on our rackspace cloud now.

What is the Builder?

What is the Builder?

The idea behind mirage is to make augmented reality accessable to all. This idea spawn the concept of a AR drag and drop builder which will be released along side our Mirage SDK. The builder will be a easy way for customers to build AR concepts by dragging and dropping video, audio, models, and images into it then creating a full AR environment. 

Building "The Builder" or at least figuring out what technologies to use really came from a website SketchFab. Like us it's a small startup except it specializes in online 3D modeling. The design and the overall usefulness served as a guide. We used a html5 heavy environment through the use of three.js as a framework. The model is then pushed to the server then served up when a targetimage is scanned just like a webpage with a link.

Technologies Used:

• WebGL
• Three.js
• Rails
• Html5

Freak API our opensource image recognition backend.

Freak Api

Freak api is one of the new list of things our very ambitious team is taking on. The basic idea is to open source image and object recognition, build a community, and the augment the world. After getting ignored by qualcomm I thought it would be good to invest in our own mobile/serverside technology for image recognition moving away from qualcomm vurforia.

Freak is feature descriptor pair with brisk that hopefully will run it all. The api will let you upload and analyse our images quickly and easily. With the renaissance in technology it has become normal for this image recognition technology to update it self pretty much every 6-10 months. We will hopefully be releasing it in thr next 3-6 months.

Server stack:

• Playframework
• JavaCV/Opencv
• Brisk/Freak

Stay tune for more details as it become availiable.