Quality Assurance
Back to LAB main
Certification Testing: Unlocking Multi-Platform Success for Your Game
Genshin Impact stands out as one of the most successful games ever in terms of both downloads and revenue since its debut in September 2020. Apart from its gameplay, a major contributing factor to Genshin Impact's success lies in its strategic cross-play capabilities across mobile, PC, and console platforms, seamlessly connecting the gaming experiences between them.
Releasing a game across multiple platforms is no novelty; before Genshin Impact, a considerable number of games had already been dual-launched on platforms like PC and mainstream consoles (Xbox/PS/Nintendo Switch) simultaneously. Moreover, with the evolution of mobile devices, mobile platforms have emerged as undeniable powerhouses that demand increasing attention from developers. Multi-platform games not only cater to diverse audiences on various platforms but also aid developers in garnering greater profits, as they tap into the expansive potential of these platforms.
However, launching games across target platforms is more complicated than “let’s develop a game and put it on PlayStation/Xbox/Nintendo Switch” or moving your hit game from the original platform to another one. To release games on an extra platform, a prerequisite is to pass certification tests.
To answer what certification testing is and examine best practices for planning and execution, we interviewed some of our Quality Assurance experts: Xing Zhang from PTW Shanghai, Bruce Lin from PTW Taipei, and Myoung Gun Kim from PTW Korea.
Xing: I am Xing Zhang, the QA team Lead at PTW Shanghai. I joined PTW Shanghai in 2003 and have more than 20 years of QA-related experience.
Myoung Gun: I'm Myoung Gun Kim, QA team manager at PTW Korea. I've been part of PTW since 2014 and have gained extensive experience in Quality Assurance over the past decade.
Bruce: I am Bruce Lin, PTW's Regional QA Manager of Asia. I have been part of PTW since 2016 and in the gaming industry for over 10 years.
Xing: Certification testing is about verifying whether a game complies with the requirements and standards for running on various gaming platforms.
The three major console manufacturers -- Sony, Microsoft, and Nintendo -- each have their own specifications for their respective gaming platforms, known as TRC, TCR, and Lot Check. If game developers want to launch their games on these consoles, they must submit their games for certification testing. Only after the manufacturer verifies that the game meets the specifications will it be allowed for subsequent release.
Myoung Gun: Certification testing is crucial for verifying that end users can play games flawlessly on their respective gaming platforms. Each platform, including consoles (PlayStation, Xbox, etc.), mobile (App Store, Google Play Store, etc.), and PC (Steam, Epic Games Store, etc.), has its own set of compliance guidelines. Failing to pass the certification test will result in the inability to release your product.
During certification testing, we make sure that all distinct technologies and regulations compatible with each console are functioning correctly. This includes verifying the effective operation of graphic rendering techniques (such as ray tracing), ensuring proper usage of unique controller haptics, adhering to varying logo usage rules, and compliance with in-game payment regulations. Additionally, for VR content, we scrutinize any aspects that could potentially impact user safety and experience.
Bruce: Other key aspects include ensuring that button callouts use the platform’s correct terminology and that the proper error messages are displayed in the right situations according to the guidelines of the platform.
Xing: Running games on mobile devices and console platforms can be vastly different in various aspects.
For example, for visual display, games run in full screen on mobile devices, but console platforms usually only allow games to occupy 90% of the screen. Therefore, developers must take this into consideration when designing the UI. Resolution and other technical aspects are also significant challenges since mobile devices and consoles have different performance capabilities. Console games require the graphical assets to fit 1080p or even 4K, whereas mobile games may not have such demanding technical requirements.
The PC platform generally has minimal certification and regulatory requirements.
Myoung Gun: Typically, PC and mobile games undergo compatibility testing to ensure they function on a range of hardware, as end users possess different specs. Meanwhile, console games are specifically designed for certain devices and do not necessitate compatibility testing.
One factor that sets different device platforms apart is their unique control features. Different gaming platforms have varied control schemes. Console games use controllers with sticks and gyroscopic sensors, while PCs and mobile devices utilize keyboards and mice, or even touchscreens. Testing for console games includes verifying different input methods.
These days, game controllers are supported on PCs and mobile devices, keyboards and mice are compatible with consoles, and mobiles have gyro sensors and vibration capabilities. However, controls still vary greatly depending on the platform the game was developed for.
Certification testing considers whether a game released on a console supports a keyboard and mouse, but it may not be an issue if the platform doesn't force support. Developers should take this into consideration, and the QA phase should also test for it.
Xing: Due to the extensive requirements of Sony’s, Microsoft’s, and Nintendo’s platforms, it is not possible to list all the specific details. However, here are some key points:
Myoung Gun: When it comes to gaming consoles, it's important to note that each platform has its own unique controller structure and naming conventions. This results in many differences, such as whether your game is compatible, what features can be shared, if live streaming is allowed, and the specific rules regarding cloud usage.
Also, the strategy for selling products varies among manufacturers. Some prioritize exclusives that are not available on other platforms when selecting games for their consoles. As a result, game developers who wish to be featured in their store must meet stringent certification standards.
At the same time, some manufacturers opt to broaden their customer base by ensuring their consoles can run a wide range of games. As opposed to the previous approach, they may opt for less stringent certification requirements to entice more game developers to their platform.
Myoung Gun: The launch of a new console often introduces cutting-edge "next-gen technology”. At PTW, we encourage testers to gain hands-on experience with the latest tech. Console manufacturers create specialized devices for game developers and testers that are not available to the public. These devices are exclusively sold to qualified individuals in limited quantities, making it challenging to acquire the necessary amount. Fortunately, we do not face this issue as we already possess a substantial number of dedicated test consoles.
The three major manufacturers of gaming consoles usually have a lengthy development process for their new devices.
Bruce: Logistically speaking, the latest generation of consoles may be hard to procure initially due to limited supplies. The solution would be to speak with both the publisher of the title and the first party to work out procurement.
Xing: For developers in China who are new to console game development, it can be challenging if they haven't done their homework in understanding the specifications of each platform and properly organizing the submission process.
Many games in China were initially designed for mobile. After gaining popularity, developers hoped to port them to console platforms. However, in most cases, those developers did not consider console compatibility during the initial game development phase. When porting mobile games to consoles, they had to overhaul many game mechanics and logic to make them compatible. This often involves revising or rewriting significant portions of the code to accommodate the hardware, controls, and other specific requirements of console platforms.
Myoung Gun: Developers in markets with limited experience in console game development, like South Korea, may encounter challenges in comprehending the platform's certification requirements during the initial stages of development.
There may be more items and tests than anticipated in the guidelines. While some documentation may be available in Korean, most recent documentation is only available in English or Japanese. This creates challenges for inexperienced developers who may struggle to comprehend the material.
And certification requirements can be subject to sudden changes, with new ones being added or removed. This can make it difficult for developers to stay up-to-date on the latest certification requirements unless they have a certified testing specialist on their team.
Bruce: Developers might not have an internal resource with the expertise to do the certification testing. Our recommendation is to work with a service provider like PTW who has certification testing expertise to get the best ROI.
Xing: Certification testing evolves alongside the rapid development of games, hardware devices, and platforms. For example, the advent of VR has led to the establishment of standards for VR games, further highlighting the importance of certification testing.
However, there is one platform that might be considered unique, and that is Xbox. Due to Microsoft's current strategy of bridging the gap between Xbox and PC, it is possible that the certification guidelines for Xbox may become more aligned with those of the PC, meaning the requirements may gradually become less stringent.
Myoung Gun: With the advancement of hardware technology, the validation process for input devices in certification tests has also evolved. Graphics and sound requirements have also become more demanding.
The rise in popularity of VR/AR/MR devices in recent years has made them more accessible and with the leaps in AI technology, it is likely that future certification requirements will be even more rigorous.
Myoung Gun: As with any kind of testing, including certification testing, having an independent group of testers separate from the developers can increase their expertise and ability to identify defects. Additionally, developers can benefit from reduced administrative tasks and costs, allowing them to focus on development.
Even larger developers with their own in-house QA departments often seek specialized companies for certification testing, particularly for console certification testing where they may have limited experience. Small and medium-sized developers may also find it more efficient to work with an outsourcer for certification testing to save resources, including manpower and costs.
Bruce: The pro is that developers and publishers do not need to maintain this headcount throughout the year – regardless of the situation, the outsource partner will have certification experts. However, the con is that the availability of the vendor varies depending on their scheduled projects. The best practice to work with an outsourcing partner would be to make a request as early in the development cycle as possible to ensure the vendor’s availability.
Bruce: Plan your certification test start date at least one month before first party certification submission. Moreover, if you think the test will result in many bugs, you may need to do some retests after you make the fixes.
Xing: I would suggest that all developers plan which platforms they intend to release their game on during the development stage, understand the certification rules of each platform as early as possible, and establish contact with suitable vendors in advance.
Myoung Gun: Collaborating with an outsourcer that employs certified testing professionals can yield significant advantages, including saving time and resources, as well as ensuring a superior level of testing proficiency.
Certification testing is indispensable for delivering high-quality, consistent, and reliable gaming experiences across diverse platforms. By focusing on performance, stability, and regulatory compliance, this process enhances player satisfaction and contributes to the overall success of a game in the competitive gaming industry. An expert team like PTW who has extensive experience in certification tests can help your games freely travel across the platform multiverse.