Play the game = test the game. Why is this statement wrong?

Today we are diving into the topic of game testing. I want to provide helpful information for QA specialists new to this domain and dispel stereotypes like “play it yourself and let your friend play too; that’s all testing.”  Also, I’ll share with you the basics of testing types, bugs, approaches, and tools to help ensure the quality of your game!

This is one of three articles on this topic. We will discuss the types of “game” bugs and examine examples andI will explore the tools that will help ensure the quality of your product. 

My name is Aliaksei Simkin, and I have something to say on the matter.

What is a video game? Based on Wikipedia, “a video game or computer game is an electronic game that involves interacting with a user interface or input such as a joystick, controller (gamepad), keyboard, or motion detection to provide visual feedback. This feedback is displayed on a video device such as a TV, monitor, touchscreen, or virtual reality headset. Video games often use audio feedback through speakers or headphones and other feedback forms, including tactile feedback.”

Gaming includes several components and the ephemeral  “fun factor.” Remember, this is, first of all, a game. Therefore, the statement “play the game = test the game” is partially true, as one of the methods of testing a game is actually playing it.

Types of game testing

There are standard approaches to testing video games. Let’s take a look at them!

Functional testing

Let’s start with the basics – functional and non-functional testing. Even though there is a lot to roam here, you don’t necessarily need a deep knowledge of games and their mechanics. The most important document you will need is the Game Design Document (GDD). All the essential nuances of your game are described here. Some developers even argue that testers don’t need to understand games to test them. They just need to follow the documentation.

Playtesting

This type of game testing is designed to understand what kind of gaming experience and emotions the gameplay will trigger in the audience. During playtests, testers give players access to an early version of the game (this type of testing is carried out before releasing the game and feature). Participants of playtesting are often ordinary gamers, relatives, friends, and in general, anyone who can take a fresh look at your product and give you feedback based on their gaming experience and impressions. Before starting, participants of the test are required to sign an NDA.

Game design testing

It includes Balance, Level Design, and “Fun Factor” testing.

• As its name indicates, balance testing is about assessing the balance in the game. For instance, one weapon is not critically stronger or weaker than another in the same class; or the easy difficulty level is more difficult than initially planned. In some cases, it is even worth thinking about the game’s meta or “balance of power”  (this is a matter of a different department; however, the QA can take notes and share them with the team!).

Sometimes you literally “balance test” your game.  

• Level design testing. This testing ensures it is possible to play through every game level without impassable places. Even the most visually beautiful levels can contain non-intentional obstacles; for example, invisible walls (when there is a model (mesh) but no textures), holes in the floor (for example, due to poor joining of planes or models). This causes gamers to fall into so-called stuck points, where a character is stranded and cannot move further.

Example of Level Design in the early stages

• Fun factor testing. Here the emphasis is on the “fun” of the game process. While some people include this as part of playtesting, others conduct it separately. 

Part of the magic of games is that they have a level of difficulty. They challenge the audience. A perfectly balanced video game might not be fun, so reviewing all aspects and levels is essential to keep the audience motivated. Mechanics and unattractive art are also crucial for the fun factor; nothing spoils the fun more than a game that doesn’t work correctly.

Visual components testing

This group includes testing 3D models/scenes, 2D (sprites), and 2.5D types of games.

• 3D model testing includes validating the correctness of the model (high and low poly) for the visual component and the number of polygons set in the requirements. It also validates:

• Whether all maps (normal, texture, occlusion, etc.) are available and used for models.
• Whether the insides of your model are visible (and if they are supposed to be visible).
• Whether the model is broken somewhere during animation (rigging and correct skinning). 

These verifications allow teams to detect issues early and develop solutions to save time and effort.

An example of working on a 3D model

• 2D and 2.5D testing work with sprites (2D) and checking 3D models for 2.5D games. Often we conduct this type of testing in fighting games, side scrollers, or roguelikes, where we usually find issues related to the rendering order of the world and heroes in isometric projects (for instance, in games such as Fallout one and two) and often the nuances encountered in general with transparent objects. 

Even though your project is in 2D / 2.5D, the game elements can interact with 3D objects and have complex effects associated with transparency. A problematic scenario, in this case, would be when an object from several sprites dissolves or appears through the alpha, and each alpha is counted separately. As a result, you will have three seemingly unrelated objects instead of one. To prevent these cases, you can render the object into a texture and use it for effect to not lose its integrity.

In terms of characters, we are not limited to game characters and their models. This should also include NPCs and enemies with their AI (AI in this context refers to how bots behave). There is already a whole field for the AI ​​testing group, including checking features such as Pathfinding, AI Spawning, AI Reaction, Detection Range / NPCs (cone of vision), etc.

Performance testing

When replacing high poly models with low poly models or even imposters, displaying polygons on loaded scenes, processing effects, etc., can drastically reduce performance and frames per second (FPS). The well-known performance testing is in charge of assessing these aspects. I would include soak testing as a part of it, as QA test specialists look for memory leaks in games. Also, within the framework of the performance testing group, it is worth mentioning stability testing, where you can detect your “favorite” things: freezes, crashes, black screens, the inability to load a level, damage to saved data, and so on.

Network testing

Bear in mind that problems can arise not only because of many models on the screen but also because of many online players in your game. Network testing can help you address these issues. Thanks to quality assurance, you can verify connection lags/drops, matchmaking, connection stability, controller input lags, and much more. 

It is worth mentioning that there might be bugs in your game due to a bad Internet connection. To detect them, you need to emulate this scenario.

Back-end testing

Now that we have covered the front-end (client), we will go through back-end testing, which often includes databases, APIs (REST APIs, for example), telemetry, etc.

• Compatibility testing

Above, I talked about performance on hardware, but I did not specify which one. Games must be optimized and transferred to different hardware of various platforms – Playstation 4/5, Xbox Serios X/S, Nintendo Switch, Steam Deck, and the most diverse and unique configurations of your personal beauty (PC). Who is responsible for executing this? You guessed it, compatibility testing. 

Many platforms have different modes of operation, for example 4K 30FPS + Raytracing, 1080p 120fps, etc. Nintendo Switch is a good example. It works in 2 modes: portable (720p) and stationary (1080p). Do not forget about the “gamepads everywhere” trend and check the variety of controllers that can work with your game on a PC or console.

• VR and AR testing

The market for VR and AR applications and games is increasingly developing. I want to highlight VR and AR testing separately because these devices have many features to fit under one size fits all. 

From my experience, it’s worth highlighting that a VR game has physical and cognitive aspects that make it more challenging to develop and test, namely, motion sickness, which occurs when using helmets.

• Compliance testing

Remember, platform holders have their own standards in addition to your quality standards, often called Technical Requirements Checklist or TRC. Usually, the QA team on the side of the platform holder is in charge of assessing such requirements as part of compliance testing. This type of testing includes in-game purchases, achievements, subscriptions, and support for streaming service features.

• Alpha and Beta testing

A similar situation occurs during alpha and beta testing. Pre-alpha, alpha, and pre-beta testing (they can be open or close) is about slicing the game at a certain point in time and checking the readiness of a particular scope at a specific stage. Even in these early stages, game testers need to use different controllers while playing and navigating your game comfortably – that’s where we come to Usability testing. There is no secret here; your game should be intuitive and pleasant to use.

• Audio testing

I am more than sure that many readers of this article are audials. Sound in games, like in movies, is one of the most potent tools for influencing the player. Audio testing is responsible for assessing such nuances and ensuring that music, sound effects (SFX), dialogues, and cues are aligned with what happens on the screen. There is specialized software for working with sound where you can find effects and replicas in automatic mode.

Equally important is to enable users to play games in their native language. Entering the global market implies extensive coverage of a potential audience. Armed with the knowledge of testing the Internationalization (I18N) and Localization (L10N), you can make your game enjoyable even on the opposite side of the world! 

• Mobile testing

We have already talked about PCs, consoles, and even VR and AR games, but we did not discuss most gamers’ devices – mobile phones. It is vital to know and understand at least the basics of mobile testing, as this market has long overtaken the console and PC in revenue. 

All types of testing mentioned above are valid for mobile games; however, it is essential to know the specifics of mobile devices and applications! This is what mobile testing is about in game-dev!

• Security testing

Do not forget about security testing. Games use accounts from different systems, including online services and accounts from your consoles (for example, you need to play Rocket League through an Epic Games account, but first, you must be logged in to Playstation Network). As players must log in to different accounts, security is essential to keep users’ information safe. 

• Game automation testing

As a final point in the “testing types” section, we will highlight game automation testing. Automation in game development is a somewhat complex process. For example, in the gameplay, we need to consider various variables, such as the character’s location and understand what is happening around them. 

However, automating UI elements or navigating screens like the main menu is simpler. Gameplay is automated by writing your bots. You can also use Image Recognition tools, suitable for automating screenshots without gameplay or even specific modules in a Game Engine.

It is worth mentioning that companies implement game testing depending on their objectives and needs. Also, standard concepts and practices can be called differently in some cases. 

See what types of testing your project or company uses, compare them with those described above, and if you have a different approach to testing games, let us know!