Since 2016, VR has been boldly breaking through into the mainstream. But for those involved in VR hardware, software, and content development, the revolution started a lot earlier.
As more tech and content companies strive to get on board the VR express, the need for well-developed functional and localization VR quality assurance has been steadily rising. Functional quality assurance testers are experts in adapting to the latest technology, and have always been able to apply their skills and experience to the latest innovations. With the launch of VR, games testing teams have been presented with some unique and difficult challenges.
It quickly became clear that effective planning, risk assessment, and creating the right test environment were essential to tackle the difficulties inherent in VR testing, and to protect the safety of testers. The physical challenges of VR have been widely publicized, and developers continue to work to overcome them. Among these challenges are hardware issues, software issues, and the physical susceptibilities of individual users.
For testers, increased exposure to VR can mean that the physical challenges are heightened.
VR games and hardware are not designed for the prolonged exposure that testers endure. Furthermore, as they are often testing initial and early versions of games or software, even the bugs themselves can cause adverse physical reactions. This is also true of the hardware that is used. Early versions of head-mounted displays had not evolved much further than the initial prototypes – the process of design refinement, aimed at comfort and enhanced user experiences, had simply not yet occurred.
Early Discoveries, Insights and Knowledge
Discomfort, calibration errors, and tracking latency hardware issues are intensified in VR. These challenges revealed unique early discoveries in VR quality assurance, and helped create a valuable body of specialist insights and knowledge. We’ll look at each condition in turn, and how it impacts on VR testing.
Category | Condition | Impact / Symptoms | Remedial Actions | Example Bug |
Physical | Cybersickness | Disorientation Dizziness Drowsiness Headaches Nausea Vomiting Fatigue Vertigo
| Immersion breaks Medical intervention Gradual exposure (new testers)
| Over-exposure |
Vergence-Accommodation Conflict | Eye strain Headaches Migraines Blurred vision Eye twitching Visual fatigue
| Screen breaks Eye tests Resting eyes
| Text stretching and snapping |
Seizures | Epileptic fits Blackouts Dizziness Migraines
| CONSULT PHYSICIAN |
Coordination | Hand-eye coordination Motor skills Balance Manual dexterity Multi-tasking Repetitive Strain | Stretching exercises Movement Stress balls Reduce number of sessions Gradual increase in gameplay durations | Prolonged controller use |
Upper Body Pain | Neck pain Backache Shoulder pain Headaches | Regular stretching Massage Ice packs | Heavy HMDs |
The Reality of Cybersickness
Cybersickness is caused when the body’s visual system registers movement, but no movement is registered in the vestibular system. (The vestibular system provides sensory information about motion, equilibrium and spatial orientation.) Subsequently, this creates a mismatch in the autonomic nervous system.
In VR, the senses are presented with the illusion of movement and, although the user may not remain still, the movement does not always correlate to what they are seeing. The more realistic the visual content, the increased likelihood of mismatch and the greater the risk of experiencing cybersickness.
In addition, particular bugs can create obvious mismatches between sight and motion. For example, if the player is looking ahead and their avatar is remaining still but the presented visual cues are of a moving landscape, this will create a clear mismatch. A higher field of view is also linked to increased cybersickness through changes and motions of optical flow. The flip side of this is that an increased field of view also connects positively with increased immersion. (Source)
There have been numerous studies into whether demographics contribute to cybersickness susceptibility. These include studies on ethnicity, age, spontaneous postural sway during natural stance (the degree of biomechanical action required to correct balance), flicker fusion frequency threshold (the degree to which an intermittent light stimulus appears steady to the average human observer), and gender. (Source)
There are no definitive conclusions to suggest that people of a particular demographic should not use VR, so consideration should still be on an individual basis. However, it’s important to stay aware of any new and developing studies.
Sometimes the phrase ‘simulator sickness’ is used in relation to VR. Simulator sickness and cybersickness are both strands of motion sickness, but they are not the same. (Source) With simulator sickness, a person will experience symptoms in a different order from cybersickness. It’s also worth noting that the severity of cybersickness was found to be approximately three times greater than that of simulator sickness.
So, what exactly is cybersickness? The intensity may vary from person to person, but the symptoms of cybersickness can include:
- Disorientation
- Dizziness
- Drowsiness
- Stomach awareness
- Headaches
- Nausea
- Vomiting
- Fatigue
Vergence-Accommodation Conflict
VR users can experience pain and discomfort in the eyes due to vergence-accommodation conflict. Testers are at a higher risk of this, due to their increased exposure to VR on a daily basis.
The key to understanding vergence-accommodation conflict is to consider the focal point of the eyes. Our eyes operate by focusing and converging on a point in space. This distance is the same for both eyes, and the brain combines this response into what is known as vergence-accommodation coupling. VR causes the focus and vergence points to be different. The eyes focus on the display screen, but the vergence distance is on the VR environment. Subsequently, this means the brain has to act against its natural coupling instinct and this can cause eye strain. (Source)
PTW has found that eye discomfort is one of the most regularly complained about symptoms of VR testing.
Seizures and Coordination
There have been concerns that VR increases the likelihood of seizures. This could depend on the design of the game being played, or the software being used - although further studies may be required.
Oculus Rift’s health and safety manual advised that about 1 in 4000 VR users may experience severe dizziness, seizures, epileptic seizures or blackouts even if they have no history of it. They also warn that prolonged use could negatively impact hand-eye coordination, balance and multi-tasking ability. (Source)
Upper Body Pain
VR can include a variety of hardware, many of which are commonly used, such as Head Mounted Displays (HMDs). HMDs come in different designs and dimensions, and there is a wide range of styles and designs on the market. The weight of HMDs can affect the user, with heavier HMDs causing greater discomfort in a shorter period of time. The movement involved when playing, combined with the use of HMDs and input devices, can result in neck, back, and even arm or hand aches. This type of Repetitive Stress Injury worsens with repeated, regular use, such as in a testing environment.
The design of HMDs varies widely, but some can cause difficulties for people who wear glasses, as the positioning of the head straps and the pressure on spectacle frames can cause discomfort across the nose and head.
VR-Specific Bugs
Accounting for 3-5% of overall bug volumes, some bugs are specific to VR games and VR experiences and can intensify all of the physical conditions we’ve described already. These could be development choices which work well in 2D environments, but cause specific physical challenges in VR.
As an example, the risk of cybersickness can increase due to specific game features - such as if a game movement occurs that the player hasn’t triggered. This might include an unexpected cutscene using the player’s camera, or if the game movement occurs too quickly, such as catapulting or transferring the player from one point to another without audio or visual cues. This can cause a change in orientation towards a specific plane, which in turn can cause significant disorientation.
Another good example involves games which use different methods of acceleration. These can result in the field of view moving erratically. As this cannot be controlled by the player, they have no focal point and are subsequently at risk of extreme disorientation.
In-game visual effects, particularly those requiring significant machine processing power, can slow down the frame-rate, causing visual ‘stutters’ and distractions. These interrupt the VR immersion experience, and can occur with even the smallest dip in frame-rate. Similarly, any rendering latency can disorientate and disrupt. Bright flashes in a game can be significantly magnified by head mounted displays, resulting in head and eye pain.
The user interface (UI) should always be within comfortable reach of the player, without them having to exert themselves, or twist uncomfortably. UIs which are held in the player’s peripheral vision, or obscured by in-game objects, can cause strain when they’re accessed regularly.