Bringing Everyday Objects to Life Using AR

Abstract

With both Augmented Reality (AR) and Natural Language Processing (NLP) being areas of very active research, this thesis addresses the combination of these technologies to create engaging AR experiences. We present the implementation of an application that detects everyday objects in the user’s surroundings and brings them to life through AR. We employ the language model GPT-3 to generate believable responses to the user’s natural language input. The objects act out their lines of dialogue using a Text To Speech (TTS) engine, offering a convincing interactive experience. A quantitative user study was conducted to assess the application’s usability and the believability of the experience. We further discuss the potentials and downfalls of using a Large Language Model (LLM) for content creation for entertainment, using our application as an example.