Good Old Games
Donate to Codex
Putting the 'role' back in role-playing games since 2002.
Odds are, something you like very much sucks. Why? Because this is the RPG Codex
News Content Gallery People Games Companies  
Forums About Donate RSS Contact Us!  

Pathfinder: Wrath of the Righteous Kickstarter Update #19: QA Automation Tools

Visit our sponsors! (or click here and disable ads)

Pathfinder: Wrath of the Righteous Kickstarter Update #19: QA Automation Tools

Development Info - posted by Infinitron on Tue 18 February 2020, 23:54:00

Tags: Owlcat Games; Pathfinder: Wrath of the Righteous

Pathfinder: Kingmaker was in a disastrously buggy state when it launched and Owlcat have pledged to do better with Pathfinder: Wrath of the Righteous. One way they aim to achieve that goal is by using automated QA testing. Today's rather interesting Kickstarter update describes some of their QA tools, which include a bot that automatically plays through the game as well as various smart logging and script inspection mechanisms to assist in fixing bugs more quickly.

When development started on Wrath of the Righteous, we decided that quality was our number one priority. We carefully calculated that to achieve our goal, we would need 23 manual testers working all day, every day from now until launch. That’s an eye-watering number of man hours to spend on testing, and that doesn’t even include management, coordination, and everything else that goes into QA. We decided that the smarter approach would be to increase the number of manual testers by a sensible amount, and alongside this, to focus on strengthening our ranks with people who could build us the extra help we needed. Yes, you heard that right! In the QA department at Owlcat Games, all our in-house tools are considered part of the team, and we named them after creatures from Golarion.

Clockwork & Arbiter

The first team member we want to introduce is Clockwork, our bot that plays through the game for us. He can:
  • Play through a specific scene. This helps us to maintain the game at the quality level already achieved and, most importantly, save time on regression testing, the most labor-intensive part of the testing process. Playing through an 80-hour game in a gazillion different ways takes a long, long time. But it’s essential work, since indirect changes can cause parts of the game to break.
  • Play according to simple behavior patterns, for example, “kill everything on sight.” These exploratory excursions will help us understand how difficult the game can be, depending on the actions players choose to take.
We’ll be expanding Clockwork’s abilities even more, so that he can find the maximum number of bug combinations and regressions. But while our automation team is developing Clockwork’s new features, he’ll be spending night after night tirelessly slaying demons (current kill count: 83,452).

So, while Clockwork is busy checking the game’s mechanics, our second bot, Arbiter, is responsible for tracking visual changes in the game world. He rants indignantly whenever he detects a missing texture or model, or sees a character holding a weapon the wrong way round, or notices a drop in performance.

The best thing about our bots is that they find bugs super efficiently, allowing us to quickly figure out what’s causing the issue and resolve it. Making fixes during the dev process increases the overall quality of the game, and also saves time that would have been spent on bug reports and fixes further down the line, since this work will stop some bugs from cropping up entirely.

Aeon, Scrivenite & Bebilith

Besides direct testing, we’re also working on reducing the time spent on finding and fixing bugs. That’s where the next guys come in.

Aeon is our comms guardian. All the messages you send us through the in-game report function pass through his true-neutral hands. Since Pathfinder: Kingmaker, he’s learned how to gather more information to understand the context around a reported error. Now the developers can log all bugs directly from the game.

Aeon also notifies the devs about critical errors, holds all the necessary information about preparing the current release, and provides it on request.

The game data consists of interrelated resources. Imagine you’re waiting for the final line of dialogue in a romance storyline. You wait and wait, but it never comes. You need this line to appear before you can finally embark on a relationship with your paramour of choice. Why isn’t the line appearing? It could be there’s a problem with the line itself, or maybe there’s an issue with the conditions required to trigger it: 3 completed quests, 10 dialogue phrases spoken at just the right time, and 18 decisions that endear you to your beloved. During debugging, the time it takes to manually study all these connections is staggering.

But with Scrivenite and Bebilith, we make this process more transparent. Scrivenite contains the entire history of changes made to resources, and Bebilith records all the connections between them. They complement each other beautifully, and together they know everything about changes made to the game code.

This data-driven duo helps us quickly see how fundamental a change is, and what percentage of the game has been touched. They also hugely speed up the process of searching for an error, the author, and when the error occurred. They let us see a clear list of changes that have been made to a specific section of the game, and whether these changes may affect the rest of the game.

On top of all this, they also seriously cut down the time it takes to finalize a game build, meaning that we can get fixes out to players in record time.
This is actually the first time we've posted about a Kickstarter update for this game that wasn't about the campaign's next stretch goal. At the rate the campaign is going right now, it'll be a long time until it gets there. We'll see what other interesting updates Owlcat can come up with during this period.

There are 3 comments on Pathfinder: Wrath of the Righteous Kickstarter Update #19: QA Automation Tools

Site hosted by Sorcerer's Place Link us!
Codex definition, a book manuscript.
eXTReMe Tracker
rpgcodex.net RSS Feed
This page was created in 0.043168067932129 seconds