Position Optimization for Intelligent AR Characters

Abstract

In recent years, augmented reality (AR) experiences that introduce virtual characters as guides or companions are attracting increasing attention. Some use cases such as museum guides or AR games are often sensitive to the spatial relationships between the scene, the user, and the virtual character. As optimal positioning of virtual characters is crucial for a positive user experience in such scenarios, position optimization methods for intelligent AR characters are in great demand. However, most prior work on placement optimization in AR environments mainly focuses on preventing 2D labels from occluding the scene of interest and important objects in view. In comparison to the label placement task, AR character positioning not only needs to accomplish occlusion- and collision avoidance, but also needs to consider several additional factors, including the 3D structure of a given scene, naturalness of the positioning and coherence in regard to the actions of the virtual character.

To tackle this challenge, we present a rule-based implementation as well as a deep learning- based implementation for this position optimization task. The rule-based implementation utilizes an angle-based algorithm to determine the angle range of areas at which the character could stay without occluding any important scene features, and computes the optimal location based on a dynamic social distance from the user. This approach achieves high accuracy in occlusion and collision avoidance. The deep learning-based implementation trains a model to predict optimal locations based on the encoded scene features and user-scene interaction characteristics. This implementation is able to provide scene-aware and human-like positioning. Both implementations are able to handle dynamic changes in real-time. A user study is carried out to examine the effectiveness of the two proposed approaches. The analysis of the user study results and feedback reveals the virtual character positioning criteria that users value the most.