RPGs give the player control of a central character usually within a Fantasy or Science Fiction environment and often include deep customization features, thousands of items, and a boundless open world. When these features are combined, not to mention amplified when adapted for Massively Multiplayer Online (MMO) RPGs, the result can be an overwhelming undertaking for quality assurance (QA) teams.

The scope of RPGs demands both exploratory and plan-driven testing, due to the number of variables teams will have to work with.

RPGs typically have a form of open world, either truly open or hub-based, which greatly increases the testing required. These open worlds will require numerous testers to consistently and constantly check for issues related to collision, textures, lighting, and rendering. This isn’t a last-minute check or a phase in testing, it’s a constant process to keep the open world bug free.

These issues can hide anywhere and every one of these that’s missed will be found by a player. Textures, rendering, and lighting issues are some of the worst bugs for this exact reason, they’ll be prevalent and obvious to anyone who encounters them, as well as often being easily reproducible and recordable. Luckily these issues usually aren’t exploitable, other than specific situations, and won’t halt a player’s progress.

However, collision issues can be some of the most irritating and exploitable. A collision issue could range from something minor to a player walking through a structure’s wall and breaking a mission’s scripting. Just imagine testing for that in the giant open worlds of RPGs with thousands of structures!

These structures also affect the game’s performance, another area where open world RPGs can find specific issues. The game must run well. Players quite simply expect it. Providing consistently high performance in an open world requires extremely thorough testing which considers all the variables RPGs typically have, and then testing these variables throughout the entire game world.

In the majority of RPGS you’ll find a laundry list of items from potions to pets and everything in between. For QA teams, this can spell a host of small but tedious checks. To guarantee full functionality of an item, every item’s effects and functions will individually need to be tested, manually or with automation. How these items then interact with one another introduces yet another set of checks that need careful consideration and testing. LQA teams will also face tedious checks here, due to the volume of possible items and strings.

Having so many items also requires extensive balancing to ensure universal consistency. Unfortunately, it’s usually not possible to balance all items before implementation. Balance checks need to be conducted throughout development, when new items or abilities are added, using every combination of items and effects. The possible item combinations in RPGs are typically extremely expansive and it’s not enough to check a couple of items together. Teams will need to check every possible combination to be sure that no edge-case issues remain, which is extremely time consuming.

RPGs typically allow players to customize their characters, introducing character stat checks, clothing and aesthetic customizations, as well as host of other specific checks. Ensuring that player characters function correctly is critical to an RPG’s success as customized characters have a more personal feel to a player than a set and standard protagonist.

Parts of this testing will be conducted using the same methods as item checks as clothing/armor options will often bring stat improvements to a character. However, as these items will be physically shown on the player’s character, a set of rendering checks will need to be implemented, per item, to guarantee that they display correctly once applied to a character. This might sound straightforward but, as is the case with items, there are heaps of edge-case checks and variables that demand testing attention, such as checking that multiple clothing items display correctly when equipped together or that the player character displays correctly in every menu.

RPGs typically boast a wide variety of NPCs (Non Player Characters) and enemies that come with their own dialogue elements, behaviors, functions, and appearances.

There are standard checks, ensuring that all enemies and NPCs display and render correctly, that they collide with objects when they should, and move freely when they don't, and that names and statistics are set correctly. On top of this, there is the important requirement to check the behavior of AI actors. This can be a very demanding part of testing, requiring a back-and-forth between the QA and development teams.

Testing the AI can also be extremely rewarding if performed correctly and attention is paid, not just to the functionality, but also to how engaging and fun these situations are. Everyone has played a game where they looked at an enemy and said, “What is this guy doing!?”, or if you're unlucky it might even be a friendly NPC!

Nothing will shatter a player's immersion like seeing a character running into a wall or behaving in an unnatural way. Player immersion can also be disrupted if testing doesn't consider the difficulty of a set encounter. If a player encounters a fight that is too hard for their level, it will almost always lead to frustration and a lack of immersion. This is particularly important in RPGs as they often allow players to engage certain encounters or enemies whenever they choose to, and if the player feels at an unfair disadvantage, they'll enjoy the encounter far less. Players don't like being frustrated and avoiding needless annoyances will always work in a game's favor. QA teams will have played every encounter numerous times and will be the best people to ask when it comes to fun, difficulty, and engagement.

But, as is always the case with RPGs, it's not just the combat behavior that needs to be checked. RPG NPCs typically have some form of non-combat routine which equally needs extensive testing. You wouldn't want NPCs walking into walls around a game, in the same way as you wouldn't want them shooting into walls. These routines often make a game feel alive and obvious bugs in these routines will quickly take away from a player's immersion.

As with items, RPGs typically have lots of menus. Whether it’s customization menus, maps, crafting screens, there are going to be a lot of options and tons of checks for menus and UI. Luckily, menus typically avoid the contextual issues of items and clothing, like equipping multiple items at the same time, unless they include specific contextual menus.

It’s not that RPGs necessarily have more intricate or expansive menus and UI than other genres, although they often do, but the sheer volume of menus, UI, and text strings that RPGs typically include make for an extensive testing effort. Consistency between menus, both linguistically and functionally, is extremely important to player enjoyment. This isn’t hard to imagine as there have been recent releases where this was missed, and every menu requires the player to press different buttons to navigate. While players will eventually get used to it, it can appear unprofessional and suggests that not enough attention has been paid to consistency in interaction.

RPGs also often feature divergent stories based on player choice, as well as often including random events. These divergent choices compound the testing required, requiring teams to playthrough the game multiple times to cover every area of the game. They can cause small dialog changes, cause different characters to appear, or change entire areas, among many other possibilities. These require thorough testing from functional and linguistic teams to ensure every option has been accounted for. Story choices often have a large impact upon how the rest of the game unfolds and ensuring that your teams account for this must be a priority. Story and gameplay choices like these also create more variables which will further increase the testing effort.

Random events don’t have quite the same effect. They don’t necessarily increase the number of variables your teams have to deal with, but they do introduce an unpredictable element that can be difficult to test due to the inconsistent nature. These random events can range from NPC dialogue to world events and require focus from both functional and linguistic teams.

Taking the time to test for all these issues is incredibly important for the title, publisher, and developer because a bug-ridden game reflects poorly upon everyone involved. It’s not hard to find major RPGs with major bugs, largely due to the size of these games and the number of variables they include. This is exactly why RPGs are also the perfect place to display your testing prowess, player focus, and RPG pedigree. RPGs with solid releases are instantly more impressive to players and easily garner more positive attention in player discussions and press coverage. You can find examples of beloved RPGs that are full of bugs, but they’re few and far between.

The important part is taking the time to do these checks. Rushing through your testing at the end of development will both take far longer and have a negative effect on your resources and employees. Avoiding crunch, as much as possible, is always a positive and not leaving any checks till the last minute will improve the overall quality of the project. Taking time for these checks will also markedly improve your linguistic quality across all support languages. Making sure that every version of the game matches linguistically isn’t just good for RPGs, it’s critical for any genre.

In summary, the biggest challenges with testing RPGs are the depth of the content and the number of variables. While a first-person shooter might have a single player character, 30 weapons, a few items, and a couple of abilities, an RPG will have multiple player characters, hundreds of possible weapons, a long list of items, and a variety of abilities which can differ based on more variables. This number of variables, and how each function with one another, compounds the amount of testing required and dramatically increases the number of testing hours needed to guarantee a successful release. RPG developers must therefore ensure considerations about the testing effort and testing strategies are factored into the development cycle as early as possible, including engagement with third party testing partners.