Discover here the second part of our review focusing on UX/UI Design for AR experiences, created with the help of Immersiv.io’s Design Lab. Get some insights about how to create AR cross-platform experiences for sports fans and the translation of an AR experience from smartphones to AR glasses. You can also access the first part of this article, describing the differences between AR on mobiles & on glasses and offering tips on AR design for smartphones.

Mobile to glasses translation: interacting with 3D content

When we had a global idea of the mobile version of the product, with user journeys, main screens and UI components, it was time to translate our smartphone UX to smart-glasses.

For the 3D virtual objects, like the mini-pitch, the translation was quite obvious. 3D objects work roughly the same on smartphones and AR glasses. But it must be taken into account that gestures and interactions are made with a controller (and laser pointer) on glasses. It means that for some of the users, there is an adaptation phase to master this controller, which may delay or hinder the experience. 

Defining those new interactions takes place essentially in the prototyping step. As we said, for mobile use, you can refer to the current standards. But you don’t have any guidelines when designing for AR glasses, so you have to define them while building your prototype. There is no other solution than knowing Unity’s basics and mastering the manufacturer’s SDKs to test the different interactions while designing AR content.

Let’s take an example: with ARISE, the user can scale a mini-pitch (for a better view), in the mobile app. Naturally, you will slide your fingers apart to scale up and join them to scale down, just like you are used to zooming in and out pictures. Now with a controller you mainly interact with your thumb. In our case, we had to rethink this feature with a new thumb gesture, letting the user select the pitch and use his thumb vertically or horizontally to scale or move the pitch. In the end, we had to inform the user of this feature with extra tips to master it, as the interaction is obviously less natural than with smartphones.

At the same time, it’s very important to prioritize information in a 3D environment. With AR glasses, you can display numerous content because the display area is bigger than on smartphones, but it’s also a trap. It’s necessary to use well the 3D environment to hierarchize content (foreground, background), focus on the area you want to push to the user, guide him to the information and send him feedback about his interactions.

Mobile to glasses translation: the flat UI overlays

Our solution is made to enhance the viewing experience of sporting events. For that, we’ve decided to insert information at key moments in a real context to add value and help the fans to better understand what they are watching. That’s why, compared to other AR app, we have added very few 3D models, to avoid distracting users from the sports competition. The idea is to make the moment enjoyable by adding contextual content, not to spend time playing or looking for AR content. 

You also have to be coherent between the screen space and the world space (your real environment) and make them complementary. It’s also the key when transposing your experience to glasses, you must take the advantage of the device, which means aerate the content, improve immersion, stage the experience. On glasses, you can’t use large flat UI overlays that stick to the camera, it would completely hide your vision. 

Therefore, we’ve spatialized all this flat information inside a moveable panel, at a right focus distance of the eyes (between 1 and 2 meters), with a continuous comfortable size to see it. The user can move and scale it at any time all around him, it is responsive to his environment and not distracting his vision. However, it was also important to implement constraints to help the user keep track of these virtual objects. For instance, we define a maximum and minimum size, a maximum distance between the panel and the user, and the panel auto-rotate to always face the user.

We’ve also added some micro-interactions to make the content and navigation more understandable. Interacting with a laser pointer can be hard when accessing an area or a button! That’s why we make the clickable areas clearer and more intelligible with user feedback (focus state, hoover state, displacement state …). It was also important to make them a little bit wider and to add animated transitions to help him keep the view on the virtual objects.

We also had to adjust colors from the previously made mobile UI. On the mobile app, we use transparent black as UI cards’ background. But on glasses, the black appears totally transparent because it is an additive display system. Large areas of very bright colors can also hurt the eyes. So we had to redefine a specific color palette, choosing transparent gray and white as backgrounds for the cards that keep a good hierarchy of information and great readability.

Learnings

These thoughts to build a common experience on smartphones and glasses made us understand and learn a lot of things. Let’s sum up.

• Imagine, prototype, test your idea/functionality early and often: The most essential is to prototype at an early stage of the product design, because a small modification can have a major impact in ensuring continuity between the smartphone and glasses experiences. And often, since we lack references, our intuitions don’t work as expected. This can make you rethink the navigation, a screen, or an item’s placement. 
• Draw storyboards to sequence the experience: too much information at the same time can overwhelm the user. Animate the UI and virtual objects, make them look alive to enhance the immersion and don’t forget to guide the user through your experience. Rework your UI style to provide a better information hierarchy and contrast. Use the space you have! You are not confined to a small screen anymore so you can make content bigger and use more white space. 

• Make different user journeys to provide an accessible experience: AR on smartphones and AR Glasses are not widely used for now. It is essential to make an appealing and easy-to-use application to democratize their use. Compensate for the knowledge deficit by creating a guided and sequenced path and support the user by providing extra tips and hints throughout his/her journey. 
• Be aware that there are some technology constraints: your app won’t be supported by all smartphones, AR glasses remain a very new device. Sometimes people will have issues with your product due to these technical constraints, devices will start to heat up, navigation will freeze, apps will crash, or lose the pitch tracking. Balance those pain points with extra care for details and attention.

Reflexion and next steps

Covid-19 obliges, people are watching sports from home, deprived of their fellow supporters. That’s why the next steps are more about the social aspects, by bringing some social interactions. For instance, users could invite a friend to the app and watch the event together using 3D avatars. 

Another challenge will be to improve the immersion and to bring back the stadium environment and its crowd with emotions, suspense, and passion by working on strong storytelling and detailed 3D assets. While doing this, we also have to anticipate technological evolutions, with the launch of new AR glasses embedding new interactions standards & a new approach in terms of UX. For instance, the HoloLens 2 has an entire hand gesture system with interactions directly from hand to virtual objects (see our Hololens 2 review).

For designers, the main goal is to democratize AR experiences and make them natural. In less than one decade smartphones went from an exclusive to a banal everyday object. With AR glasses, we can imagine a similar, even faster, scenario. In the near future, when attending a match, you will bring your AR glasses to enjoy the game with live interacting data or exchange with your friend who stayed at home, sharing them the electrifying atmosphere of the stadium.

Remember to read the first part of this review: Designing AR experiences for mobile & AR glasses (part 1).