Why Agile was the Right Choice for Our Runtastic QA Team

_{Written by Jacob Smith, Software Test Manager}

As the Quality Assurance (QA) team at Runtastic, our goal has always been to ensure the highest quality product for our users. And with a continuously expanding feature set, our mission to do so proves both fun and challenging. It requires us not only to reevaluate our current processes, but also to adopt new solutions to maintain high quality within our apps. That’s why when the whole company transitioned to Agile, we welcomed the challenge and saw it as an opportunity to improve.

The Old Days

Before our transition to Agile back in 2016, our approach to QA was manual testing after feature development outside of the development teams. Before ever receiving a test build, we would get a set of specifications for the given feature and test case creation would begin based on those specifications. These test cases would then be used for feature testing, which only began after development concluded. The first feature test cycle also served as a review phase for the test cases. Once any critical issues discovered in the first round of feature testing were checked and resolved, a second feature test cycle commenced.

The Problem

Giving feedback on functionality and usability is the most important thing we can do as a QA team to deliver a high quality product. And doing that as efficiently as possible is imperative for us to succeed. However, we increasingly realized we needed to improve our processes to deliver that feedback in a good way. For example:

• Providing test cases for every possible use case, including one-offs and edge cases
• No early access to the feature – the first test build was only provided at the end of feature development
• No validation of acceptance criteria from QA
• Mid-development phase spec changes
• Differing ideas of implementation for the same specs
• Long feedback loop

Not only that, we found that any impediments encountered early in the development cycle usually resulted in sacrifices made later to meet release deadlines. This usually translated to a sacrifice in quality – shorter, rushed testing phases, stressful releases and ultimately, bugs in production.

That isn’t to say we encountered every single one of these issues for every release. And even when we did, we still managed to ship great quality products. But we didn’t want to ship just great quality products – we wanted to ship the best quality products. Therefore, we needed to improve.

From Bug Detection to Bug Prevention

For us, improvement came in the form of Agile. After the transition happened and our workflows normalized, we found that many of the issues we faced in previous years were addressed with Agile’s iterative development cycle. The most obvious change we noticed is that QA simply happens faster and more often.

With a dedicated Software Test Manager in each Squad, quality is a topic from the very beginning. Starting with the very early stages of a feature idea all the way to the detailed work required by each User Story, the Software Test Manager is included in the process. And it’s no longer a QA-only topic – it’s a Squad responsibility with QA at the forefront. The close collaboration and collective ownership of the product has led to shorter loops, quicker feedback and easier communication. Many issues are already encountered during acceptance testing and can be quickly addressed by the developers on the spot, as the code is still fresh in their memories. This has also decreased the occurrence and criticality of bugs encountered in later test runs.

Additionally, with the introduction of automated tests, we substantially decreased the amount of effort we invest in scripted testing. In contrast to the old days of covering all possible use cases with scripted tests, they’re only considered when absolutely necessary with our current process. And the time saved in creating them is now used to enhance preparation work, ultimately shifting the focus from bug detection to bug prevention. Furthermore, the time saved in executing scripted tests has also allowed us to diversify our test portfolio to include exploratory, user-flow and mob testing. Overall, this has led to smoother, less stressful releases and fewer bugs slipping into production.

Learnings

Of course, change does not come without growing pains. While many of the weaknesses we encountered with the old processes were immediately addressed, we were also faced with new and exciting challenges. The following learnings were vital in the success of our transition to Agile:

• Understanding the Role of QA Within the Squad

Our goal as QA remains the same, but the method of reaching that goal has evolved. Where once our input wasn’t included until one of the final phases of the development cycle, it is now present from start to finish. Learning how best to take advantage of that and insert quality driven objectives was a process, but ultimately brought us to where we are now.

• Squad Collaboration

Developing trust within a group of people does not happen overnight, especially if those people have not worked closely together before. But for Agile to succeed, a certain level of trust is necessary within the Squad. We’ve found reaching that level takes time and effort, but that investment results in healthy collaboration between all roles within the Squad.

• Clear & Concise User Stories

Finding the perfect medium between too much and not enough has been a learning process when it comes to User Stories. And we’ve discovered writing clear, concise, and accurate User Stories is a fundamental criteria to developing high quality products. That way, when the time comes to test a User Story, the functionality is obvious to everyone in the squad.

Conclusion

Here at Runtastic, one of our core values is to embrace flexibility and constant change, and this principle was especially crucial when we transitioned to Agile. Looking back, it’s easy to say it was the right choice for us as a QA team. QA isn’t just a checkpoint at the end of the development lifecycle anymore – our holistic approach now allows us the opportunity to influence processes and insert quality throughout the entire development process. The results have helped us reach our goals of both ensuring the best quality product for our users and maximizing efficiency at the same time.

Has your team also transitioned to Agile? What has your experience been like so far? We’d love to hear about it in the comments. Also, did reading this pique your interest? Then check out our available positions – we’re always on the lookout for like-minded people.

***