Regression testing is an essential function in software quality assurance whether you are working on a path breaking new product, or some age old legacy system. It’s the only way to ensure that your enhancements, bug fixes or new features have not broken any existing code, and to avoid those 2 AM fire alarms (God forbid), when your yearlong project crashes the moment it meets a production environment. Depending on your organization size, culture, budget etc., you probably have a dedicated team that automates the testcases you marked for regression, and there is an automation suite that is diligently run before any production rollout. Whether it is current and has full coverage is a topic for another day.
So traditionally we have been used to this concept of automation very late in the cycle, generally after a battery of other tests have been run and your AUT( application under test) has withstood them. With Agile methodologies, it’s a whole different ballgame. Agile advocates short sprints or iterations, and each of these iterations have to produce a potentially shippable piece of functionality. It means that your features are coded and tested in every possible way. As this functionality builds up every subsequent iteration, you test more and more because you are integrating everything that came before and also testing it. Doing all this manually is a herculean task, and your friendly Agile coach, always around to lend you a helping hand, has urged you to use test automation. But this may not really be feasible.
In Agile projects, the codebase can and does change pretty rapidly from day to day if not from hour to hour. It means you could have a lot of bugs and are almost always rewriting or updating your testcases. It could also mean an application that is not very stable, and automating something that changes every day may not really seem to be a value add. Yet it is as formidable a task to manually test more and more functionality every subsequent Sprint. The ‘Sprint+1’ approach is the answer, according to this white paper on HP.com.
Agile automation: A primer on test automation in Agile projects (http://h20195.www2.hp.com/v2/GetPDF.aspx/c02879554.pdf) suggests that test automation should lag behind, but not more than for one iteration. So we assume that ‘story 1’ is completely coded and tested etc. in Sprint 1, and is automated in Sprint 2. The assumption here is that ‘story 1’ is stable at the end of ‘Sprint 1’. Some of the salient points and/or questions that I can think of based on agile projects I worked on are –
1) Even if a certain story is ‘done’ at the end of one iteration, it could always have changes and reappear as a new story, hence requiring more work on it and rework as regards test automation.
2) Test automation should certainly be a separate item on the product/sprint backlog. This will force all stakeholders to recognize and acknowledge that test automation is a necessary part of the project and requires a chunk of time and resources. It will add visibility in terms of estimates and burndown.
3) Team members can concentrate on manually testing unstable portions and automating stable functionality. A different person could be doing each of these and either can give full attention to the task at hand, since the actual tasks or sub-tasks for achieving either will be different.
4) Test automation can or should be a story in each iteration with specific task cards for each particular feature, story etc. that is to be automated.
I can go on, but I suppose you get the point. Every Agile team will be facing slightly different circumstances in terms of their company culture, product, technology, people etc. And each of you will need to come up with something that best fits your situation, try it for one sprint/iteration at a time, and keep improving on it based on your retrospectives.
So how are you handling test automation on your Agile project? Do you have a 100% automated regression suite? Do you think you are saving time using test automation? Or are you still happy using manual means for regression? I am eager to know what you think.
No comments:
Post a Comment