Back to LAB main
An established interactive games publisher faced major technical and scheduling challenges when releasing a virtual reality (VR) title. We worked in partnership with the client on this ground-breaking title to improve functionality, quality and meet the target for time-to-market.
The client has a strong reputation for innovative and ground-breaking work. They have won an abundance of awards, and received global critical acclaim. Being a frontrunner in VR games, the client wanted to launch their VR title at the same time as new VR hardware.
The company places a strong emphasis on functional quality assurance. To give their new products the best chance of success, they required testing for the following features:
As a VR testing project, this presented a variety of interesting challenges to PTW’s functional testing team, including:
The PTW Approach
Working in close partnership with our client, we were able to deliver bespoke VR Quality Assurance services. Our functional testing team worked as an extension of the client’s internal teams, reporting directly to the company’s internal Quality Assurance team.
Key areas of working ensured the success of the projects. These included:
The team members went through self-learning and training programs to give them a full understanding of the products and hardware. This included extensive VR immersion tests to allow the team to become familiarized with prolonged periods of VR testing, and establish the optimum duration for screen breaks, down time and recovery. Risk Assessment
Risk assessments were carried out prior to assigning staff. These considerations included establishing if any of the testers had a history of migraines, eye conditions, epilepsy, neck or back issues. We also looked at other situational factors, both medical and non-medical. Testing Environment
We created a specific VR testing environment with a larger space for movement, light blockers for direct sunlight, and well-organized placement of equipment and cables. Test Design and Execution
Early on in the project, the team adopted an exploratory testing method. Each feature was then tested before it was added to the title specification, focusing on stability and connectivity issues when multiplayer games were accessible. As the game evolved, the team had to fragment the game into different sections to enable them to complete thorough testing.
Towards the end of the project, the team were able to formalize testing by breaking the different features of the game into test cases. Our team create the test cases, in collaboration with the client. They also began developing methods that had never previously been used on projects involving an outsource team. These included using Visual Studio to open and analyze crash dumps to assist the development team in determining the cause. Once the defects were fixed, they were re-tested and regressed around the fixes to ensure that there were no breaks created during fixes.
Client Reviews and Feedback
Periodic reviews and regular customer feedback were in place to ensure that the program was on track and the client was satisfied with PTW's work. As our team reported directly to the client's internal Quality Assurance team, and we operated as a fully aligned partnership, the lines of communication were refreshingly open.
A great result The meticulous testing, supported by robust test processes and continuous improvement, helped in achieving exceptionally high quality deliverables. The functional testing team discovered more than 2,000 bugs, in addition to bugs discovered by the client's in-house teams. Some of the technology and tools we utilized included:
The success of the project results in a number of positives, including:
Outsourcing quality assurance also reduced the client's staffing overhead, including measures to cover shift rotations through overtime, particularly where there were workflow or scheduling bottlenecks. This method of working also freed up the client's in-house teams, allowing them to divert to priority tasks elsewhere on the project.