Software QA FYI - SQAFYI

Thoughts on Test Automation in Agile

By: Rajneesh Namta

Working in the time box of a scrum sprint does present some challenges - especially if your team is building a product from scratch. Sprint after sprint you have to add new features and ensure that whatever you built previously continues to work. Having an automated testing framework, which takes care of both system and integration tests, adds a lot of firepower to such a team. It not only acts as a safety net against regressions caused by new development, but more importantly frees up a lot of precious developer and tester time - allowing them to focus on the things they do best.

In this article, I intend to share with you some of the test automation techniques our team successfully implemented on a recent project – the results of which have proven to be a huge asset. Although it took us quite some time to reach where we are now, it’s been well worth the investment and the efforts are paying off in big way. We are now able to build, integrate, test and deploy our application software with production-like quality in a production-like environment every single day. While we had our share of good days and bad days, we managed to learn something new through the experience and apply that learning to make things better.

Our team is very happy with the results we are seeing from applying test automation techniques to our project. Applying these techniques has allowed us to consistently add new features to the product at a very brisk pace with every sprint and has proven to be crucial in finding and fixing critical issues we faced as a result of regressions.

Here are a few of the real life lessons we learned which should keep in you in good stead if you begin adding test automation to your projects.

Start Small

The process you follow when creating automated tests is very similar to the process you follow for creating the software being tested. It involves a fair bit of design, coding and testing of its own to get it working correctly. So just like the application itself, automated tests are best developed incrementally - adding new tests and features to the automation framework over several different sprints. It’s important not to aim for the perfect “it-can-do-everything” test framework right at the start, as it would never materialize. Balance the cost vs. ROI well and come up with a bare minimum working solution at the start. Working tests, just like working software, are useful; they build confidence and get everyone excited about the progress being made. Successes, even small ones, make it easier to bring everyone on-board – especially when the test automation solution you have created actually runs and proves to be of real value to the team.

Test Automation Backlog
Maintain a test automation backlog for your project that contains all needed automation tasks and identified improvements. If you then target a few items from the backlog every sprint, in no time you will start to see the new regression test suite taking shape. Occasionally, stories from the test automation backlog may require dedicated developer time to implement and consequently some buy-in from the product owner in order to proceed. However, it should not be difficult to convince the product owner about the value of such stories if everyone on team is committed to quality.

A test automation backlog could contain a prioritized list of items such as:
* Parameterize the test environment for test execution.
* Integrate with Continuous Integration.
* Enhance reporting mechanism.
* Provide an option to attach error logs in notification emails.
* Collect performance metrics for workflow scenarios.
* Add tests to check for concurrent execution of critical test cases.

Tools Are Just the Means And Not The End
The tools and frameworks you use to achieve test automation are not the real goal of your testing efforts. If you focus on the big picture, the real goal is to support new development efforts by providing rapid feedback to the team. This helps keep everyone informed about the current state of the project so that interested stakeholders can make informed decisions. Since tools and frameworks are only a means to achieve a much broader end, it is important to not get obsessed with the new tools and lose sight of the ultimate goal.

It is also important to keep your tests and test data independent of the selected test automation tool as much as possible. Creating tests with hard-coded test data, system configuration, and properties makes them difficult to maintain. And in the long run, strong coupling with the test data makes it difficult to change your test tooling midway through the project if you were to run into any unanticipated issues.

Full article...

Other Resource

... to read more articles, visit

Thoughts on Test Automation in Agile