Did you Hear That Wink? Non-Visual, Non-Verbal Social Interaction Patterns from the Audio Game Community

This seminar explored non-visual social interaction design in VR by examining the audio game community, which creates immersive multiplayer games using spatial audio for blind and visually impaired users. Presenter Brandon Biggs conducted a Delphi study with the top five active audio game developers to create a comprehensive inventory of nonverbal interaction patterns for non-visual VR experiences. The research revealed that while academia has explored basic haptic VR tools (haptic gloves, canes, styluses, 360 treadmills) and auditory elements (binaural audio, head-tracking, sonar sticks, auditory beacons), these efforts remain minimal compared to visual VR development. Meanwhile, the thriving audio game community on audiogames.net has developed sophisticated conventions across strategy games, FPS, RPGs, and MUDs like Swamp, Survive the Wild, Shades of Doom, A Hero’s Call, and Alter Aeon, demonstrating that spatial audio creates experiences as immersive for blind users as visual VR does for sighted users.

The study identified key design patterns across nine categories of social interaction: movement (using joystick controls with footstep and collision sounds rather than physical gestures that lack feedback), camera position (always maintaining first-person audio perspective), facial expressions and gestures (combining short earcons with text descriptions like “[earcon] Brandon winked at you”), multi-avatar interactions (exaggerated sounds plus speech for physical contact), avatar mannerisms (subtle, optional periodic sounds), avatar appearance (speech descriptions with customization options), avatar-environment interactions, full-body animations (preset emotional expressions with spatial audio), and conventional communication (speech messages with menu navigation). The research emphasized that blind players strongly dislike VR’s typical reaching-and-gesturing interactions because they provide no tactile feedback, preferring gamepad or keyboard controls with clear auditory indicators of movement start/stop and collision.

Critical accessibility recommendations included allowing users to access their own familiar screen readers and synthesizers rather than forcing proprietary systems, ensuring all content uses semantic HTML that screen readers can parse (avoiding WebGL/Canvas “black boxes”), implementing keyboard-first navigation, making sounds and messages optional/customizable, and providing ARIA live regions or tools like TOLK for screen reader integration. The presenters noted that while 95% of blind gamers use Windows (where most audio games run), mainstream VR platforms like Mozilla Hubs have resisted adding basic navigation sounds despite years of advocacy. Future work includes analyzing interface conventions in top audio games and building platforms incorporating these standards, though major engines like Unreal and Unity have been largely unresponsive to accessibility needs, while Godot shows promise with recent semantic tooling additions.

Generated from notes via Anthropic’s Sonnet 4.5 LLM