The testing pyramid – consider these in your QA projects

Simply put, testing is just one activity within QA, aimed at gathering facts about the quality—or lack thereof—of your product.

Assuring quality, however, is much more. It includes everything from planning and documentation to review practices and coding. It’s about the development process and metrics you use to track your progress.

While testing isn’t all there is in QA, testing and approaching the game through The Testing Pyramid can help point out areas that need development. The Testing Pyramid concept was created to examine how much emphasis should be put on the different levels of testing. Sounds confusing? Let’s dig in together.

The testing pyramid

Some 15 years back, The Testing Pyramid concept was created to describe the amount, focus, and coverage each level of testing should achieve. In all simplicity, unit testing is at the base of the pyramid. On top of that comes integration tests, and at the top, UI tests (also known as end-to-end testing, or system test, depending on the context).

The thinking behind this is that the lower in the pyramid you go, the more reliable your tests are. At the same time, the higher in the pyramid you go, the more difficult, slow, and less covering your tests are likely to be. Basically, you should build test coverage by testing e.g. permutations fast and easy in the unit test, and only run basic and edge cases in the UI testing.

In mobile gaming we need more focus on the lower areas of the pyramid

In traditional IT, the pyramid is quite steep and the top-level is far too big, to reflect the fact that even there is too much focus on UI tests. In gaming, the pyramid tends to be pretty much upside down, i.e. you have a tiny amount of unit tests, maybe some integration tests and a huge set of manual UI tests (typically also referred to as manual QA).

However, UI testing is of course also crucial, but we need to build the test coverage in place by creating, maintaining and focusing also on unit- and integration testing. Only through this, we can actually put the focus of system testing – where it should be.

In UI testing you should then focus on the complete system and end-user experiences instead of hunting down tricky bugs that randomly appear due to a variable for a function deep down in the architecture not being initialized correctly.

The pyramid is just a starting point

The Testing Pyramid itself doesn’t solve anything, but understanding the concept and evaluating how you are applying this concept in your projects is something that should always be kept in mind.

Not everything is possible to cover with unit tests, and yes, for a game it can be really hard. That doesn’t mean that we shouldn’t look into this. The more effort is put into determining what can be done and what should be done, the more everyone will most definitely get to enjoy the benefits.

Interested in QA and testing? Read also about the importance of planning and documentation – the often missed key stages in QA of the game industry.

Henry Lunabba is a Project Lead at Metacore. He’s spent almost half of his career in quality assurance (QA for short) within project, service, and people management—and now, in games. In this blog series, he will guide us through the lessons he has learned about managing (and sometimes, just surviving) chaos when developing new systems and launching new products.