Efficiently entering text without the traditional keyboard in XR headsets has been a tough nut to crack, yet it’s essential for boosting productivity in immersive tech. To tackle this, a group of researchers has compiled an exhaustive list of text entry methods, thoroughly examining their advantages and drawbacks. By sharing this resource openly, they hope to ignite fresh ideas and innovations in this promising field.
Recently, I sat down with an insightful guest, Massimiliano Di Luca, who heads the VR Lab at the University of Birmingham in the UK. Di Luca, now an Associate Professor of Psychology and Computer Science, made waves at Meta with his groundbreaking work on hand inputs and haptics for VR. His excellence hasn’t gone unnoticed; he’s been honored with the ACM SIGCHI 2025 award for his pioneering work in XR interaction, especially the Android XR framework.
As VR and AR platforms continue to evolve, the need for seamless text entry becomes more pressing. Whether crafting emails in a virtual office or mingling in the metaverse, effective text input is a must. Seeing this need, our team, including researchers from esteemed institutions like the University of Copenhagen and Google, launched the XR TEXT Trove. This initiative catalogs over 170 unique text entry techniques tailored for extended reality, offering a treasure trove of methods from academia and industry alike.
The techniques are meticulously organized with a comprehensive array of 32 codes, such as input devices, the body part used for input, concurrency, and haptic feedback modality. Additionally, there are 14 performance metrics evaluated, including words per minute (WPM) and total error rate (TER). Such detailed categorization offers an extensive view of the current landscape of XR text entry techniques.
One significant insight from our findings is that the speed of text input is closely tied to the number of input elements, be it fingers, controllers, etc. Speed gains similar to typing on a traditional keyboard are only attained with multi-finger typing. Each extra input element improves speed by roughly 5 WPM, as illustrated in our data visualization.
Moreover, the role of haptic feedback, external surfaces, and fingertip-only visualization in enhancing typing performance can’t be understated. Typing on surfaces, as opposed to mid-air, not only improves efficiency but also eases the physical strain, minimizing the risk of what’s humorously referred to as “Gorilla Arm Syndrome.”
Interestingly, conventional keyboards remain unparalleled in terms of speed, likely due to their efficiency despite the steep learning curve. We speculate that the future of typing in VR, faster than on computers, might be achieved by reducing the typing distance using AI and machine learning—similarly to how swipe typing revolutionized single-finger typing on smartphones.
The XR Text Trove is a pivotal resource that sheds light on virtual and augmented reality text input methodologies. By providing an organized, user-friendly database, we aim to support researchers and developers in crafting more effective, user-friendly text input solutions for our immersive future.
The importance of this work for the XR community is emphasized in our accompanying paper: “To support XR research and design in this area, we make the database and the associated tool available on the XR TEXT Trove website.” Look for the full discussion at the upcoming ACM CHI conference in Yokohama, Japan.
Several contributors to this project also played a role in creating the Locomotion Vault, a similar effort focused on VR locomotion techniques, helping researchers and designers fast-track the development of new solutions.