Interesting and Innovative Projects & Experiments
Here at CrossComm, we regularly come across opportunities to experiment with upcoming technologies, create interesting proof-of-concepts, and explore different internal product ideas. The following is a collection of such experiments; feel free to reach out if you want to start conversation about anything we are working on!
We are very privelaged to have the opportunity to work with one of a small number of prototypes of the Meta 2 AR Headset. We have created a number of demos with this impressive hardware, but one of our favorites is this custom UI for navigation within our demo app. Please enjoy this short demonstration!
When we started our internal labs initiative in 2015, we founded it with a focus on Artificial Intelligence as well as Augmented Reality, because in order to have a future with great AR, we'll need great AI - in particular, great computer vision that understands the semantic context of our surroundings.
We have developed a demo leveraging Facebook's recently open-sourced DeepMask and MultipathNet computer vision frameworks to recognize common household objects, and a demo using TensorFlow neural network locally on the device to recognize what sees and offer a translation of what you're looking at courtesy of Google Translate. Imagine this kind of app running on a pair of smartglasses, allowing you to immerse yourself in another language while staying within your everyday context. This is just one of the examples of how AR and AI will converge to offer a whole new breed of useful applications for work and life.
Previously we introduced ARKit and outlined some of its core features and abilities. In this video, we'll explore how ARKit allows for the creation of interactive virtual objects.
In this demo app, a user can place an virtual keyboard on a horizontal surface. Once placed, the user can interact with this virtual keyboard using standard tap and swipe gestures that mirror the actual experience of using a keyboard. The dials and sliders on the keyboard can also be manipulated by dragging one's finger, and the keys depressing by tapping. Interactivity is essential for an engaging experience- and this example demonstrates how virtual buttons, control panels, interactive widgets can go beyond 2D heads-up displays and integrate into our real world surroundings in a spatial manner.
ARKit is a software framework built into the upcoming iOS 11 that offers Augmented Reality functionality and capabilities to iOS apps. Technically, ARKit is a highly impressive work of engineering, allowing apps running on any iPhone as recent as the 6s or even the SE to recognize flat horizontal surfaces through the phone's camera and track the device's changing position and orientation relative to its surroundings.
The following ARKit demo app is programmed to respond to the user's screen touches by generating 3D computer-generated spiders. We've programmed these 3D spiders to walk along the surface of the floor creating the illusion that these spiders reside, or at the very least are interacting with, our actual world. Notice the perspective of these computer-generated spiders changing with the movement of the phone in a smooth, natural fashion - so that the illusion is consistently maintained.
Augmented Reality can make magic happen. This lab experiment overlays video on top of still images within a 3D space, creating the same visual effect seen in the "Harry Potter" movies where still photos come to life.
We think this technique could be used to great effect within the context of museums, storefront advertising, amusement parks and entertainment venues- anywhere where visual content has a story to tell. And while impressive on a smartphone, we believe this effect will really shine when paired with wearable AR devices like smartglasses and headsets.