PTW - How PTW introduced automation testing

Home
Services & Industries
Services & Industries

All Services

Customer Experience

Quality Assurance

Localization

Audio Production

Engineering
About
News & Events

Careers
Contact & Locations
LAB
English

English

Canadian French

Japanese 日本語

Korean 한국어

Simplified Chinese 简体中文

How PTW introduced automation testing for an online games company, with winning results

PTW Engineering Case Study

We were tasked by a client to increase the efficiency and efficacy of their quality assurance testing. The client specializes in delivering software for online gaming, with over 100,000 customers daily. Before working with PTW, the client had only performed manual testing. Perhaps not surprisingly, they’d found this to be ineffective and inefficient. A huge effort was needed for validating their application logic under different platforms. To provide quicker release times, with complete test coverage, automating their testing had become a necessity.

The Requirements

The client urgently needed automation to enhance their quality assurance testing across different platforms, and to keep their business competitive. PTW took complete ownership of the automation testing solution – from Planning and Framework to Script Development and Maintenance, Environment, and Execution.

The Challenges

The first challenge was automating a flash game. This was a big task in terms of identifying the right automation tools, framework and solution. The creation of a cost-effective solution, with open source tools, helped to reduce the overall costs. We launched the project by creating a Proof of Concept, to identify a suitable tool for the customer.
Most of the existing data was dynamic, apart from the login credentials. This meant it was technically dif?cult to physically populate the same data again for automation needs. We solved this issue by creating scripts for handling dynamic data at run-time, to minimize dependency on inputs.
The existing permutations and combinations were huge in number, making it extremely dif?cult to validate manually. To overcome this, we designed an automation framework to execute scripts 24/7, and calculate the probability of results over a period of time.
The test environment setup was challenging, as it involved multiple devices and operating systems. Creating and utilizing an automation lab to target multiple devices simultaneously allowed greater coverage, and far greater efficiency.
Too much time was required for writing complex test scripts, given the dynamic status of the games at run time. We resolved this by developing a separate utility to recover the application from these undefined events

The PTW Approach

We involved all stakeholders in the project, and the communication and engagement with the existing client team was excellent from start to finish.

We followed our robust test automation approach for this client, which we’ve created in line with industry standards. Our approach has evolved to meet the needs of our clients across the globe, who operate in different domains with different automation requirements.

Our approach is structured into seven clear phases:
1. Objective
High level understanding of software

2. Tools
Evaluation of tools and platforms

3. Planning
Assessing the modules/functionalities for automation

4. Framework
Design and development of framework support application functionalities

5. Setup
Version Control and test lab setup

6. Test
Design and execution of automation scripts

7. Maintenance
Scripts and framework maintenance

Some of the technology and tools we utilized included:
We put in place periodic reviews and regular client feedback to ensure that the program was on track and the client remained satisfied with our work.

The client was thrilled with the results that our solution provided, which included:

Selenium
Java
C#
Appium Studio
Jira
SharePoint
J-Meter

The result was a complete overhaul of the client’s system, which provided numerous benefits:

We delivered a cost-effective solution, with higher Return on Investment (ROI).
Automation testing reduced 80% of the effort and time spent by the manual testing team.
We achieved the same level of compliance on all the supported environments.
We increased con?dence on release to the client’s target consumer markets, thanks to high quality releases and greater test coverage
We were also able to help the client with performance testing and analysis to optimize the application performance

The client was delighted with the following return on investment:

Parameters	Manual	Automation
Time to Design Test Cases	24 Hours	24 Hours
Time to Automate Test Cases	N/A	1,200 Hours
Total Time of automation	N/A	1,224 Hours
Time to execute a full test cycle	750	750
Total time of Testing per release	7,500	1,500 Hours
Saving per Release	N/A	6000 Hours
Releases per Year	1 Release	1 Release
Benefit per year	N/A	6,000 Hours per year
Timer saving per year ((Benefit minus total time of automation))	N/A	4,776 Hours per year
ROI % (Benefit divided by investment)	N/A	390%

Get in touch