The QA story – and how best to portray it on your sprint backlog

Quality assurance and/or quality control is an integral part of the software development process. It ensures you conform to your requirements, meaning that you are delivering what your customer wants, and isn’t that what we are all striving toward in a nutshell?

Agile advocates test driven development or TDD, and many Agile or Scrum teams use it. But that’s a big topic on its own. I want to focus this post on how you portray testing or QA on your sprint backlog, whether it’s a TDD type activity, or manual testing or automation testing or any aspect of the testing lifecycle be it planning, design, execution, defect tracking, process improvement etc. These are tasks that you would typically have even on an Agile/Scrum team if you follow a modicum test management process. These need to be done for each story on your sprint backlog. E.g if your story is about creating a way to login to your system, or adding an item to a shopping cart, you still want to write high level and detailed testcases for each requirement pertaining to this story, then have a task for executing these testcases, have a task for defect logging if necessary etc. You will probably need tasks for setting up any initial conditions such as test environment or specific data setup. You will need tasks for obtaining approvals for your testcases, or following up on defects etc. You also might need tasks for other admin type activities such as setting up your test tools, assigning appropriate roles to team members, ensuring that your developers are set up right in the defect management systems so that you can assign your open bugs to them etc. etc. etc. The list is endless and you may do some or all of these depending on your organization, your team and the minimum artifacts you need to produce from an audit or compliance standpoint. I have not even touched on tasks relating to regression, integration etc.

Lets pause a second to take a deep breath. Does this seem like a lot of work? QA or testing almost always is. It takes a lot of due diligence to ensure sufficient test coverage and it’s the last step before you release your baby into the big bad world. So how do you portray all this on a sprint backlog? Create cards for each and every small thing, or just clump some things under a ‘miscellaneous’ heading?

This is what my experience has been. As you work on successive sprints, you start realizing the myriad small tasks or things that need to be done to reach the point of say, executing a testcase, and you gradually start creating separate tasks for them. This may appear to be overkill initially but it has its advantages. You increase visibility into what you do, and make the entire team aware of the nuances of a typical QA function. You make it easier for your team to understand your estimates – it is easier for everyone to follow several tasks that are 2-4 hours than having one number such as 20 hours or 40 hours that most will think you pulled out of your hat. This also builds greater understanding in everyone’s mind about the impact any changes to requirements or code will have on QA. So you get it – the advantages are manifold.

Getting down a bit further to brass tacks, I have generally had specific tasks such as designing and documenting testcases, executing them, adding defects as part of specific stories, and then I have a separate story I call the ‘QA story’. All my administrative tasks go here e.g. resource onboarding, tool setup, work requests, status calls, mentoring, knowledge transfers etc. etc.

One big advantage of this is that I already know what most of the tasks are going to be each sprint, and it makes it easier all around during part two of the sprint planning meeting. Each tester can focus on the stories they are going to work on, with only one or two ‘new’ tasks they may need for a specific story. This saves time being an almost repeatable process. Of course, it has taken me time and several different projects to come up with this.

What do you think of this post? How do you portray your QA tasks on your sprint backlog? Do you have different tasks each sprint, or do you feel that you are doing similar things for each story? I would love to know your views and experiences.

Sprint plus one – is it the answer to your test automation woes?

Regression testing is an essential function in software quality assurance whether you are working on a path breaking new product, or some age old legacy system. It’s the only way to ensure that your enhancements, bug fixes or new features have not broken any existing code, and to avoid those 2 AM fire alarms (God forbid), when your yearlong project crashes the moment it meets a production environment. Depending on your organization size, culture, budget etc., you probably have a dedicated team that automates the testcases you marked for regression, and there is an automation suite that is diligently run before any production rollout. Whether it is current and has full coverage is a topic for another day.

So traditionally we have been used to this concept of automation very late in the cycle, generally after a battery of other tests have been run and your AUT( application under test) has withstood them. With Agile methodologies, it’s a whole different ballgame. Agile advocates short sprints or iterations, and each of these iterations have to produce a potentially shippable piece of functionality. It means that your features are coded and tested in every possible way. As this functionality builds up every subsequent iteration, you test more and more because you are integrating everything that came before and also testing it. Doing all this manually is a herculean task, and your friendly Agile coach, always around to lend you a helping hand, has urged you to use test automation. But this may not really be feasible.

In Agile projects, the codebase can and does change pretty rapidly from day to day if not from hour to hour. It means you could have a lot of bugs and are almost always rewriting or updating your testcases. It could also mean an application that is not very stable, and automating something that changes every day may not really seem to be a value add. Yet it is as formidable a task to manually test more and more functionality every subsequent Sprint. The ‘Sprint+1’ approach is the answer, according to this white paper on HP.com.

Agile automation: A primer on test automation in Agile projects (http://h20195.www2.hp.com/v2/GetPDF.aspx/c02879554.pdf) suggests that test automation should lag behind, but not more than for one iteration. So we assume that ‘story 1’ is completely coded and tested etc. in Sprint 1, and is automated in Sprint 2. The assumption here is that ‘story 1’ is stable at the end of ‘Sprint 1’. Some of the salient points and/or questions that I can think of based on agile projects I worked on are –

1)  Even if a certain story is ‘done’ at the end of one iteration, it could always have changes and reappear as a new story, hence requiring more work on it and rework as regards test automation.

2)  Test automation should certainly be a separate item on the product/sprint backlog. This will force all stakeholders to recognize and acknowledge that test automation is a necessary part of the project and requires a chunk of time and resources. It will add visibility in terms of estimates and burndown.

3)  Team members can concentrate on manually testing unstable portions and automating stable functionality. A different person could be doing each of these and either can give full attention to the task at hand, since the actual tasks or sub-tasks for achieving either will be different.

4)  Test automation can or should be a story in each iteration with specific task cards for each particular feature, story etc. that is to be automated.

I can go on, but I suppose you get the point. Every Agile team will be facing slightly different circumstances in terms of their company culture, product, technology, people etc. And each of you will need to come up with something that best fits your situation, try it for one sprint/iteration at a time, and keep improving on it based on your retrospectives.

So how are you handling test automation on your Agile project? Do you have a 100% automated regression suite? Do you think you are saving time using test automation? Or are you still happy using manual means for regression? I am eager to know what you think.