All About Testing: How Quality Assurance Works at Runtastic

_{Written by Hebatalla Elsayed, Software Tester}

After Runtastic moved to cross-functional teams in 2016, QA Software Test Managers joined the Squads and started actively participating in the whole process of feature development, from definition to release and everything in between. The transformation was both exciting and challenging for us as QA: on one hand, it helped detect bugs early or even prevent them, as quality is achieved and maintained every step of the way. On the other hand, it also meant additional responsibilities and expectations. In this post, we will reflect on how the Runtastic QA team is working now, following up on our last post on why we think the transition was important.

Software Test Managers

Feature Planning

When the Squad is brainstorming for the next feature to be implemented, Software Test Managers always keep the quality perspective present. In Three Amigos meetings that follow the brainstorming, they bring the possible edge cases to the discussion. These cases are especially important when the Squad is preparing mind maps for user flows in the new feature. As much as this helps in bug prevention, it also paves the way for maintaining more specific, higher quality User Stories with clear Acceptance Criteria.

Feature Development

Once the feature’s User Stories are ready for development, Software Test Managers start writing detailed Feature Scenarios which will be pushed later for automated tests. Below is an example feature scenario written by a Test Manager:

For the scenarios that cannot be covered by test automation, Software Test Managers can still use them for writing test cases with instructions for manual testing. The scenarios are also kept by the Squad as a source of truth for product documentation.

Feature: Login Component - Login

  As an existing user,
  I have to pass through Login,
  In order to use Runtastic apps

Background: Given user is registered

@login-email
Scenario: User logs in via email
And user is on the Email login screen
And user entered valid credentials
When user taps to continue
Then user is logged into the app

@login-facebook
Scenario: User logs in via Facebook
And user is on the Login screen
And user tapped on Facebook login
And user entered valid credentials
When user taps to continue
And user passed the permissions screen
Then user is logged into the app

And then the Testing Starts!

• Acceptance Testing :Acceptance Testing starts with the first available build. These tests are only passed if the Acceptance Criteria for each User Story are met and no blocking issues are found.
• Mob Testing: When all User Stories of a feature are done and the required functionality is implemented. The Software Test Manager organizes a Mob Testing session for the Squad to test the build, where Android developers test the iOS build and vice versa. After finishing the mob testing session, developers fix any bugs encountered and prepare the first release candidate.
• Sanity Test: After the first release candidate is ready, the Software Test Manager makes a quick Sanity Test to make sure it is ready to be tested by Software Testers, for us, these are QA team members who are not assigned to Squads, but help with manual testing.

Software Testers

To apply our four-eyes principle, which ensures that our builds are tested from different perspectives, Software Testers work is always separated from the Squad. Software Test Managers create Test Requests for each platform describing the purpose of the test, the changes that were implemented in the build, and any other necessary information for the Testers to execute the test.

Feature Testing

The first Test Request is either:

• A Feature Test with specific test cases covering only the new feature, and not examining any other existing features in the app.
• Or an Exploratory Test where the tester freely navigates the whole app, with some sort of focus on the new feature.

Software Testers run those tests, report bugs and document their feedback on the new functionality.

User Flow Testing

An additional User Flow test is done by testers when applicable, in order to give subjective feedback about the look and feel of the feature. Ideally this test answers the question of “Does the feature work well for our users?” rather than “Does the feature work as designed?”. Testers adopt the perspective of specific personas while doing this User Flow testing, so that they are able to come up with new ideas for improvement.

All findings are then reviewed by the Squad before the next and final Test Request gets ready for testing – the Smoke Test.

Smoke Tests

The Smoke Test is an extensive test cycle that covers the most important features of an app. It is how we do regression testing and, therefore, has to be done before each release to make sure that the critical app functionality is still intact after a new feature or fix is implemented. After the Software Testers finish the whole Smoke Test, their findings are again reviewed by the Squad, and upon successful conclusion of this test cycle, the Squad then decides on shipping the new version to the app stores.

Conclusion

With a closer look into the workflow described, we can observe that the QA team’s scope has been extended so much that it is no longer about testing only: Software Test Managers now actively participate in all stages of feature development. Moreover, collaboration with Product Owners, as well as Developers, is significantly broadened, resulting in better mutual understanding, continuous feedback and, above all, better product quality.

***