Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception

arXiv:2611.07612 · eess.AS · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "Auditory and Visual Virtualization in 6DoF XR".

Dev: Here's the gist of the paper. Extended reality (XR) enables controlled yet naturalistic multisensory experiments, but its validity depends on whether virtualized cues are perceptually plausible and lead to behavior comparable to the real world The single most informative sentence about the paper's subject and finding is that "Overall, the findings demonstrate the strong potential of current 6DoF XR systems for interactive perceptual research, while revealing that task-specific behavioral validation may be necessary".

Rosa: System Implementation Details The study utilized a custom 6DoF XR system combining liveRAZR real-time room acoustic simulation and binaural rendering with Unreal Engine-based visual rendering The system employed Unreal Engine 5.1.1 for visual rendering and OpenXR and Open Sound Control (OSC) for data distribution For audio, liveRAZR 0.6.0 was used for real-time room acoustic simulation and 6DoF binaural rendering The XR hardware included an HTC Vive Pro Eye HMD without headphones or a HTC Vive Tracker 3.0 attached to the employed headphones combined with a Samsung Galaxy Tab S3 for AAR contexts Binaural signals were presented via AKG K1000 extra-aural headphones connected to a Sennheiser EW IEM G4 wireless audio transmitter The entire system ran on a Windows 10 Pro PC with 64 GB of memory, a 16-core AMD Ryzen 9 5950X CPU, and an NVIDIA GeForce RTX 3070 GPU.

Dev: Plausibility Assessment Findings The plausibility experiment compared real and virtual sound sources in audio-augmented reality (AAR) and virtual reality (VR) The system produced plausible binaural rendering in both AAR and VR, with higher plausibility in VR.

Rosa: The results showed that the mean d′ of the VR condition was 0.16, compared to 0.41 for the AAR condition.

Title and authors: Rosa: So we're looking at this paper called "Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception," and it basically tackles whether these virtual sound cues actually feel real <ref:2611.07612#pg1,Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic>.

Dev: Right, the title itself is pretty direct about testing how much auditory and visual virtualization affects how believable things are in a six-degree-of-freedom extended reality system.

Rosa: Yeah, what interests me is that they aren't just looking at whether it sounds good or not, but specifically how this virtualization impacts something like distance perception when you're moving around.

Taro: I’m curious if they actually test this in a way that shows how the system handles things when the world gets weird, like when someone is walking and the sound comes from a strange direction.

Rosa: They do, and what they found is that while it’s hard to tell if it sounds real—the plausibility assessment—the actual distance judgments got messed up in a specific way depending on whether you used audio or just visuals to fake things.

Dev: That’s the core finding, right? They showed that any form of virtualization generally made performance worse and took more effort when trying to judge distances dynamically.

Rosa: Exactly, and the interesting part is that auditory virtualization caused a systematic bias where people tended to think sound sources were closer than they really were.

Taro: That means if you’re relying on virtual audio cues while moving, you’re systematically underestimating how far away those things actually are.

Dev: And visual-only virtualization was different; it didn't cause that kind of directional bias toward nearer sources, but it still reduced overall accuracy in the distance estimation task.

Rosa: So what they’re saying is that auditory virtualization and visual-only virtualization had different effects on distance perception, and they didn't just add up to create a bigger problem in full virtual reality.

Taro: It seems like the way the audio was virtualized was more impactful for distance estimation than just having visual stuff layered on top.

Dev: That points toward needing a more nuanced approach when designing these systems because you can’t just assume that if one modality works, it helps the other modality perform better in a complex task.

Rosa: So what they suggest next is that you need to move beyond just making things sound plausible and start testing how those virtual cues actually affect the specific perceptual processes you care about.

Taro: Which means if your goal is to study something fundamental, like how the brain perceives distance, you need to validate the system for that specific process rather than just checking if it looks right.

Dev: It sounds like they’re saying plausibility and actual behavioral accuracy aren't exactly the same thing, which is a big distinction for anyone building these kinds of interactive systems.

Rosa: That’s what they’re highlighting, and that means researchers need to use task-specific validation when they want to generalize what they learn about perception.

Taro: So the implication for us is that just making the sound look convincing isn't enough; we have to prove it helps people perform the specific task we're studying correctly.

Dev: Exactly, and that means we can’t just rely on a high sensitivity parameter like d prime alone to say everything about how real the experience feels.

Rosa: So, to wrap up this paper, the authors are suggesting that while these 6DoF XR systems have huge potential for interactive research, they need to be validated against the actual perceptual and behavioral processes they are trying to study <ref:2611.07612#pg1>.

Taro: I just want to emphasize that if you're building something for real-world use or serious cognitive study, testing it in a controlled lab room isn't enough; you have to see how it handles the messiness of a real environment.

Dev: And from an engineering standpoint, this means our safety cases need to account for these kinds of systematic biases that aren't just about latency or tracking errors but about how the sensory input is being virtualized.

Rosa: So, if you’re working on XR audio design right now, this paper shows that you need to think beyond just achieving a high plausibility score and focus on the specific behavioral outcome of your system.

Taro: We should probably start thinking about how we can build validation layers into our systems that specifically check for those types of distance biases before we even deploy them widely.

Dev: That makes sense; it shifts the engineering focus from just making the rendering loop run smoothly to ensuring the perceptual consequences of that rendering are sound.

Rosa: So, the paper "Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception" ultimately tells us that for immersive systems to be useful in research, they need more than just looking convincing; they need to perform correctly on the specific perceptual tasks we set up <ref:2611.07612#pg1,Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic>.

The paper's summary: Rosa: The authors found that while these six-degree-of-freedom systems can be really useful for interactive research, whether it *feels* real depends on whether the virtual cues actually make sense to us.

Taro: So it's not just about if the sound is loud or not, but if our brains are tricked into believing something is closer or farther away than it really is.

Dev: Exactly. They tested plausibility—how convincing the audio and visuals are—and then they tested dynamic distance perception while we were moving along a line.

Rosa: And what they showed was that auditory virtualization caused a specific bias, making us think sound sources were systematically nearer than they actually were, and visual-only changes didn't cause that specific directional bias.

Taro: That’s interesting because it means the way you fake distance with audio is different from how you fake it with visuals.

Dev: Right. The study showed that any kind of virtualization generally made performance worse and took more effort when trying to judge distances dynamically, but auditory virtualization was the biggest problem there.

Rosa: And the authors actually ended up saying that all these effects—the sound stuff and the visual stuff—don't just add up to make a bigger mess in full virtual reality.

Taro: So, if you’re building an immersive system, you can’t just assume that making it look plausible is enough to guarantee the distance judgments will be correct.

Dev: That’s what they are suggesting. Plausibility tells you if the environment looks consistent, but task-specific validation is still necessary when you want to study something fundamental about how we perceive distance in real life.

Rosa: It means researchers can't just check a box saying their system is plausible; they have to prove it works for the actual perceptual process they are studying, whether that’s distance estimation or something else.

Taro: So, the implication is that we need to validate these immersive systems not just for overall plausibility but for the specific things we intend to study or train with them.

Dev: It shifts the focus from just making a cool simulation to ensuring that when you use that system in a lab, it actually produces accurate behavioral results for whatever you're testing.

Rosa: And that leads us to asking how we can build systems that check for those specific biases before we even deploy them widely.

The paper's improvements: Rosa: So, looking at what they found, the authors are suggesting that just checking if something sounds plausible isn't enough to prove its real-world worth in research.

Dev: They're saying that plausibility can confirm the environment is free of obvious artifacts, but task-specific validation is still necessary when you want to study fundamental perceptual processes or generalize behavioral findings.

Taro: That means researchers have to move beyond just making things look convincing and start testing how those virtual cues actually affect the specific perceptual process they are trying to study.

Rosa: Exactly. They’re pointing out that plausibility and actual behavioral accuracy aren't exactly the same thing, which is a big distinction for anyone building these kinds of interactive systems.

Dev: So, if you’re working on an immersive setup, you can't just rely on a high sensitivity parameter to say everything about how real the experience feels across different tasks.

Taro: That means we need validation layers into our systems that specifically check for those types of distance biases before we even deploy them widely.

Rosa: And that implies that when we look at these 6DoF XR systems, the goal shouldn't just be achieving a high plausibility score, but ensuring they perform correctly on the actual tasks we set up <ref:2611.07612#pg1>.

Dev: It means shifting the focus from just making the rendering loop run smoothly to ensuring that the perceptual consequences of that rendering are sound for whatever you’re testing.

Taro: So, if you’re building something for serious cognitive study, you need to prove it works on those specific tasks rather than just checking if it looks right in a controlled room.

Rosa: That's the point. The future work they suggest is to examine whether similar behavioral results emerge with other room types or more detailed acoustic models, but that’s still lab work for now.

Dev: And they also flag a limitation: the study was done in one big lab room with specific geometry, which might mean those findings don't hold up everywhere.

Taro: So, if you're building something like this today, you need to think about how it handles the world misbehaving outside that perfect lab setting.

Rosa: That’s what we need to focus on next—how do we build validation for these systems when they aren't perfectly controlled environments?

Conclusion: Rosa: So we're wrapping up on "Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception <ref:2611.07612#pg1,Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic>." It boils down to this—the paper shows that even when a system sounds convincing, if it’s not validated for the specific task you’re doing, its performance might still be off.

Dev: Right. The main implication is that we can't just chase high plausibility numbers in these six-degree-of-freedom systems; we have to check if they actually work for the perceptual process we care about.

Taro: I agree with that, it means the system needs to be validated against what it’s supposed to do when things get messy in the real world, not just checking a score in a clean lab setting.

Rosa: That’s right. The paper emphasizes that plausibility and behavioral validity aren't interchangeable; researchers need task-specific validation when they want to generalize what they learn about perception.

Dev: It’s about making sure the controls engineer knows what the safety case is, not just that the rendering loop runs smoothly with low latency.

Taro: I just want to push on how this changes things for autonomy research; it means we need validation for failure modes, not just for when everything works perfectly in a straight line.

Rosa: So, if you’re building an immersive system, the lesson is that you need to go beyond checking if it looks right and prove it performs correctly on the specific distance estimation task.

Dev: That means we have to integrate that validation layer into our design from the start, so we're not just focusing on rendering fidelity but also perceptual accuracy under dynamic conditions.

Taro: It makes sense for autonomy; if a system misjudges distance because of how it virtualizes sound, that’s a failure mode you need to address gracefully.

Rosa: Exactly. We can't just assume the sensory input is fine just because the visualization looks good on screen or in the simulation.

Dev: So, for anyone working with XR audio right now, don't just aim for a high sensitivity number; focus on proving that auditory virtualization doesn't introduce systematic biases in distance judgment.

Taro: And that leads us to thinking about those future directions they mentioned, like testing this on people with hearing difficulties or in different room types.

Rosa: We’ll be talking about those more next time, but for now, remember that "Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception" is a great starting point for figuring out what makes a system truly usable in research <ref:2611.07612#pg1,Auditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic>.

Johannes M. Arend, Christian Scheer, Melissa Ramírez, Stefan Weinzierl, Fabian Brinkmann

Aalto University · Technische Universität Berlin · Institute of Computer and Communication Technology TH K¨oln - University of Applied Sciences

eess.AS

Submitted: 2026-11-05

Updated: 2026-11-05

Comments: Submitted to a peer-reviewed journal

Code: https://github.com/gamlj/gamlj

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 77/100

The gist: Extended reality (XR) enables controlled yet naturalistic multisensory experiments, but its validity depends on whether virtualized cues are perceptually plausible and lead to behavior comparable to…

Key concepts

6DoF XR Systems
These are immersive virtual reality systems that track a user's position and orientation in three dimensions plus six degrees of freedom (movement). The study used a custom system combining live room acoustics, binaural audio rendering, and visual rendering to create an interactive environment where sound is placed realistically around the user.
Plausibility Assessment
This experiment checks if virtualized sound cues feel realistic compared to real sounds. The researchers found that while the system produced plausible renderings in both VR and AAR, the auditory virtualization condition resulted in significantly lower sensitivity, meaning participants were less able to tell real from virtual sound.
Dynamic Auditory Distance Perception (ADP)
This task measures how accurately users judge the distance to sound sources while moving. The study showed that auditory virtualization caused a systematic bias toward perceiving sound sources as closer than they actually were, which changed depending on how far away the speakers were placed.

Terminology

Summary

Extended reality (XR) enables controlled yet naturalistic multisensory experiments, but its validity depends on whether virtualized cues are perceptually plausible and lead to behavior comparable to the real world The single most informative sentence about the paper's subject and finding is that Overall, the findings demonstrate the strong potential of current 6DoF XR systems for interactive perceptual research, while revealing that task-specific behavioral validation may be necessary.

System Implementation Details

The study utilized a custom 6DoF XR system combining liveRAZR real-time room acoustic simulation and binaural rendering with Unreal Engine-based visual rendering The system employed Unreal Engine 5.1.1 for visual rendering and OpenXR and Open Sound Control (OSC) for data distribution For audio, liveRAZR 0. p. 9

Where the paper says it

1 for visual rendering, as well as for gathering and distributing tracking, control, and response data using OpenXR and Open Sound Control (OSC). In particular, Unreal received listener tracking data from the XR hardware, rendered the visual scene accordingly, and transmitted listener and sound source position/orientation data via OSC to liveRAZR 0.

6.0 was used for real-time room acoustic simulation and 6DoF binaural rendering The XR hardware included an HTC Vive Pro Eye HMD without headphones or a HTC Vive Tracker 3. p. 9
Where the paper says it

Regarding XR hardware, we used an HTC Vive Pro Eye HMD (without the headphones) together with an HTC Vive Controller 2. 0 to provide responses in VR contexts, or a HTC Vive Tracker 3.

0 attached to the employed headphones combined with a Samsung Galaxy Tab S3 for AAR contexts Binaural signals were presented via AKG K1000 extra-aural headphones connected to a Sennheiser EW IEM G4 wireless audio transmitter The entire system ran on a Windows 10 Pro PC with 64 GB of memory, a 16-core AMD Ryzen 9 5950X CPU, and an NVIDIA GeForce RTX 3070 GPU. p. 9
Where the paper says it

0 to provide responses in VR contexts, or a HTC Vive Tracker 3. 0 (attached to the employed headphones; see below) combined with a Samsung Galaxy Tab S3 for user input in AAR contexts.

Plausibility Assessment Findings

The plausibility experiment compared real and virtual sound sources in audio-augmented reality (AAR) and virtual reality (VR) The system produced plausible binaural rendering in both AAR and VR, with higher plausibility in VR. The results showed that the mean d′ of the VR condition was 0. p. 5

Where the paper says it

Participants compared the real and virtual loudspeakers based on seven room acoustic qualities, including source distance, using a 100-point scale. The results showed that, for most of the examined rendering options, the perceived source distance was comparable to that of the real loudspeaker, indicating good overall agreement for ADP between the real and virtual environments.

16, compared to 0. p. 0
Where the paper says it

In two experiments with 45 participants, we investigated how auditory and visual virtualization affect plausibility and dynamic auditory distance perception (ADP). A plausibility experiment compared real and virtual sound sources in audio-augmented reality (AAR) and virtual reality (VR), while a locomotion-based ADP experiment assessed distance-judgment accuracy, signed error, and task effort across four virtualization conditions.

41 for the AAR condition. p. 6
Where the paper says it

RQ3: How do auditory and visual virtualization individually affect the fundamental perceptual processes of dynamic ADP in a locomotion-based task? RQ4: Do the effects of auditory and visual virtualization combine additively in the fully virtual condition, or is performance primarily determined by one modality?

A paired-samples t-test confirmed that the VR condition yielded significantly lower sensitivity than the AAR condition. p. 5
Where the paper says it

Their results showed that both individualized and non-individualized BRIRs yielded ADP comparable to that of real sound sources in a congruent listening environment. However, incongruence in the form of a significantly lower reverberation time in the reproduction room than in the recording room compressed perceived distances for at least a subset of participants, indicating room divergence effects [35, 36] that must be considered for XR audio rendering.

The mean d′ of the VR condition was slightly below the conservative threshold of d′ = 0. p. 2
Where the paper says it

In the experiment introducing the paradigm, Lindau and Weinzierl [16] yielded a very high degree of plausibility for binaural rendering with 1DoF, with participants mostly guessing whether a sound was real or simulated. The authors proposed that a binaural AAR system can be considered plausible if the mean d ′ across subjects is significantly smaller than a threshold of d ′ = 0.

1777, yielding a Bayes Factor (BF) of 6. p. 2
Where the paper says it

The authors proposed that a binaural AAR system can be considered plausible if the mean d ′ across subjects is significantly smaller than a threshold of d ′ = 0. 1777, as determined by a minimum effect test of the null hypothesis.

11 in favor of the null hypothesis that it does not exceed this threshold. p. 2
Where the paper says it

The authors proposed that a binaural AAR system can be considered plausible if the mean d ′ across subjects is significantly smaller than a threshold of d ′ = 0. 1777, as determined by a minimum effect test of the null hypothesis.

Dynamic Distance Perception Results

The dynamic ADP experiment assessed distance-judgment accuracy, signed error, and task effort across four virtualization conditions Accuracy decreased with increasing loudspeaker distance across all conditions, with the fully real condition (RA/RV) yielding the highest proportion correct. Auditory virtualization led to a systematic bias toward nearer loudspeakers that increased with distance, reflecting a compression of perceived distance that increased with source distance. p. 1

Where the paper says it

In the ADP task, any form of virtualization reduced accuracy and increased effort, although performance remained high overall. Virtual audio produced a bias toward nearer perceived source distances, whereas visual-only virtualization reduced accuracy without a significant directional bias.

Visual-only virtualization did not produce a comparable directional bias, mainly reducing overall accuracy without introducing a significant bias toward nearer or farther loudspeakers. p. 1
Where the paper says it

In the ADP task, any form of virtualization reduced accuracy and increased effort, although performance remained high overall. Virtual audio produced a bias toward nearer perceived source distances, whereas visual-only virtualization reduced accuracy without a significant directional bias.

Conclusion on Virtualization Effects

The study demonstrated that any form of virtualization leads to a general decrease in performance and an increase in effort in dynamic ADP. Auditory virtualization led to a systematic underestimation of source distance, while visual-only virtualization primarily increased uncertainty. p. 1

Where the paper says it

In particular, systematic perceptual evaluations of six-degree-of-freedom (6DoF) systems, in which users can move freely within a virtual space while the audio and visual renderings update continuously in real time, are lacking. Furthermore, the perceptual implications of auditory and visual virtualization in such 6DoF systems on perceptual quality measures, such as plausibility, and fundamental perceptual processes, such as dynamic auditory distance perception (ADP), remain poorly understood and have yet to be investigated.

Importantly, performance in the fully virtual condition was primarily defined by auditory virtualization rather than by an accumulation of auditory and visual virtualization effects, suggesting that effects of auditory and visual virtualization do not add up. p. 6
Where the paper says it

RQ3: How do auditory and visual virtualization individually affect the fundamental perceptual processes of dynamic ADP in a locomotion-based task? RQ4: Do the effects of auditory and visual virtualization combine additively in the fully virtual condition, or is performance primarily determined by one modality?

Implications for XR Research

The results suggest that plausibility alone may not be sufficient to establish real-world perceptual equivalence. Plausibility assessment can confirm that the environment is free of obvious artifacts or inconsistencies, but task-specific validation remains necessary when the goal is to study fundamental perceptual processes or generalize behavioral findings. p. 1

Where the paper says it

Overall, the findings demonstrate the strong potential of current 6DoF XR systems for interactive perceptual research, while revealing that task-specific behavioral validation may be necessary. Keywords: Extended Reality, Virtual Acoustics, Binaural Rendering, Multisensory Perception, Plausibility, Auditory Distance Perception 1 Introduction Extended reality (XR) technologies, including augmented, mixed and virtual reality (AR, MR and VR), have had a significant impact on the design and conduct of perceptual multisensory studies and cognitive experiments [1–3].

The research indicates that immersive systems should be validated not only for overall plausibility but also for the specific perceptual and behavioral processes they are intended to study, diagnose, or train. p. 1
Where the paper says it

Overall, the findings demonstrate the strong potential of current 6DoF XR systems for interactive perceptual research, while revealing that task-specific behavioral validation may be necessary. Keywords: Extended Reality, Virtual Acoustics, Binaural Rendering, Multisensory Perception, Plausibility, Auditory Distance Perception 1 Introduction Extended reality (XR) technologies, including augmented, mixed and virtual reality (AR, MR and VR), have had a significant impact on the design and conduct of perceptual multisensory studies and cognitive experiments [1–3].

Limitations and Future Directions

The study was conducted in a single, large laboratory room with specific acoustic and geometric properties. Simplifications such as the shoebox approximation of the geometry and non-individualized HRTFs may have contributed to discriminations between real and virtual sound sources. p. 5

Where the paper says it

, 0DoF) based on measured individualized or non-individualized BRIRs, which prevented visual and dynamic auditory cues. Their results showed that both individualized and non-individualized BRIRs yielded ADP comparable to that of real sound sources in a congruent listening environment.

Future studies should examine whether similar behavioral results emerge with other room types, more geometrically detailed acoustic models, and individualized HRTFs. p. 6
Where the paper says it

Second, they performed a dynamic, locomotion-based ADP task under four virtualization conditions—real audio / real vision, real audio / virtual vision, virtual audio / real vision, and virtual audio / virtual vision—allowing us to systematically examine modality-specific and combined virtualization effects. The main contributions of this paper are as follows: • We present the technical implementation and calibration of a real-time 6DoF XR system for perceptual experiments, combining real-time room acoustic simulation and binaural rendering with a visually detailed virtual replica of a large laboratory room.

The locomotion-based ADP task constrained participants to walk along a straight line in front of the loudspeakers Future work may examine dynamic ADP with richer movement patterns and compare different response methods within the same 6DoF XR system. p. 6
Where the paper says it

Second, they performed a dynamic, locomotion-based ADP task under four virtualization conditions—real audio / real vision, real audio / virtual vision, virtual audio / real vision, and virtual audio / virtual vision—allowing us to systematically examine modality-specific and combined virtualization effects. The main contributions of this paper are as follows: • We present the technical implementation and calibration of a real-time 6DoF XR system for perceptual experiments, combining real-time room acoustic simulation and binaural rendering with a visually detailed virtual replica of a large laboratory room.

The study should also assess the applicability of such a state-of-the-art 6DoF XR system to individuals reporting functional hearing difficulties. p. 6
Where the paper says it

Moreover, no prior work has systematically investigated auditory and visual virtualization in a unified, controlled paradigm, nor has any study assessed dynamic ADP in plausible, state-of-the-art, 6DoF XR environments with real-time room acoustic simulation. Addressing these gaps is essential to understanding whether recent XR systems can accurately reproduce everyday multisensory spatial hearing and can be effectively used in behavioral and perceptual research.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Develop an AI system capable of predicting individual perceptual requirements for AAR versus VR audio rendering based on plausibility testing results, as shown by Sensitivity was significantly lower in VR, suggesting that introducing a virtual visual environment reduced participants’ ability to distinguish between real and virtual sound sources.

  2. Implement a dynamic auditory distance perception module that can predict the effect of modality-specific virtualization on distance estimation accuracy and effort during locomotion tasks, specifically identifying that auditory virtualization led to a systematic bias toward nearer loudspeakers while visual-only virtualization mainly reduced overall accuracy without introducing a significant bias toward nearer or farther loudspeakers.

  3. Create an AI validation layer that assesses whether the plausibility of an XR audio system translates into task-specific behavioral equivalence, suggesting that plausibility and behavioral validity are related but not equivalent and that researchers should use task-specific validation remains necessary when the goal is to study fundamental perceptual processes or generalize behavioral findings.

Abstract

Extended reality (XR) enables controlled yet naturalistic multisensory experiments, but its validity depends on whether virtualized cues are perceptually plausible and lead to behavior comparable to the real world. To the best of our knowledge, this is the first study to perceptually evaluate a six-degree-of-freedom (6DoF) XR system that combines real-time room acoustic simulation, binaural rendering, and an audio-visually matched virtual replica of a large laboratory room in terms of both plausibility and behavior. In two experiments with 45 participants, we investigated how auditory and visual virtualization affect plausibility and dynamic auditory distance perception (ADP). A plausibility experiment compared real and virtual sound sources in audio-augmented reality (AAR) and virtual reality (VR), while a locomotion-based ADP experiment assessed distance-judgment accuracy, signed error, and task effort across four virtualization conditions. The system produced plausible binaural rendering in both AAR and VR, with higher plausibility in VR, indicating that perceptual requirements for XR audio depend on the visual reference frame. In the ADP task, any form of virtualization reduced accuracy and increased effort, although performance remained high overall. Virtual audio produced a bias toward nearer perceived source distances, whereas visual-only virtualization reduced accuracy without a significant directional bias. These effects did not add up in full VR. Overall, the findings demonstrate the strong potential of current 6DoF XR systems for interactive perceptual research, while revealing that task-specific behavioral validation may be necessary.

Related papers