Do’s and don’ts of starting up test automation

(Begin of language: English)

Ah, the promised land of test automation. So much money, time and energy saved! When starting up test automation, your goal is efficiency and frequent checks! Yet, a lot of automation efforts are far from efficient. And that’s too bad, because there really are a lot of good things in test automation – if done right.

In a pension project, we implemented a solution where health declaration were automatically accepted, denied or put in a queue for manual processing, depending on four different scores. While the solution was implemented, I made an automated data-driven test suite, testing all the different combinations and ranges of the scores.

The final test, after a month of development, took me 10 minutes. It actually only took me 5 minutes, but the project leader couldn’t believe her ears, so she made me run it again.

That project was perfect for test automation. But not every project is as straightforward. I’ve collected some Dos and Don’ts for you if you’re considering implementing automated test suites.

(End of language: English)

Don't...

Automate all your manual tests

(Begin of language: English)

Manual tests are amazing, and will find problems you never imagined existed. But the manual approach does not always translate directly to automated checks. Consider making new scenarios specifically for automation.

(End of language: English)

Leave the automation job to one specialist

(Begin of language: English)

While it’s a good idea to hire an automation consultant to get everything started, make sure that the knowledge is shared with others in the project. One day, (This link open in new tab) the consultant or key person will be gone, and others will have to take over the test suite.

(End of language: English)

Automate everything

(Begin of language: English)

There must be a reason that a test scenario is automated. It has to give some sort of value to the project. Start with the mundane tasks that you are doing every day anyway. That’s where you’ll find the first, immediate payoff of your automation efforts.

(End of language: English)

Automate for automation's sake

(Begin of language: English)

Think about what you need, and what you need right now. If you aim for 80%-100% automation in all projects just because “that’s the big thing and the CEO wants it” or “it sounds good when presenting to the board”, you’re wasting time and money.

(End of language: English)

Buy the first, the best automation tool

(Begin of language: English)

Find a tool that fits your needs, and make sure that the tool can actually do what it promises. Don’t be persuaded by a seller to buy an expensive program that is really smart right now, in this specific project or on this specific platform. And don’t force other projects in your company to use it if they don’t need it.

(End of language: English)

Think in "passed" and "fail"

(Begin of language: English)

You will get cases that “fail”. That doesn’t mean that there’s an error in there. Don’t translate a “Failed” flag to a bug, you need to figure out WHY the test failed. Also consider whether the sharp “Passed/Failed” categories are the right way to report the automated runs. Maybe another classification would be better suited?

(End of language: English)

Run everything, every time

(Begin of language: English)

Think about what the purpose of the test run is. If you don’t need to test a specific area – don’t. Only test functionality that needs testing. Testing everything else is a waste and clutters your reports with irrelevant data. Sure, you can have constantly running checks, but make sure their purpose is clear.

(End of language: English)

Forget to test your tests

(Begin of language: English)

You should get used to the idea of testing your test. An automated test scenario shouldn’t be released unless you’ve seen it fail – several times. Run the test under circumstances where it surely will fail, so you can count on that it will when put into action.

(End of language: English)

Underestimate the effort it takes to keep an automated suite running

(Begin of language: English)

Having an automated test suite is not something that can be started up and then run on its own. There are constant changes in your systems, and the suite needs to be updated and expanded. Don’t leave the responsibility to that one technical tester in your company.

(End of language: English)

Do...

Break down tests into independent scenarios

(Begin of language: English)

It’s easier to figure out where the problems and errors are if your suite consists of short, specific test scenarios. Don’t test it all in one scenario. If you have to look at every single thing that is being checked in the scenario, troubleshooting will be VERY hard. Keep it short and simple.

(End of language: English)

Make the automated test suite a common project effort

(Begin of language: English)

Why not let everyone, not just testers, have an influence on and contribute to the suite? Developers, project manager, product owners.. They all have different reasons for wanting an automated test suite, and different ways of contributing.

(End of language: English)

Let people know of the results of the automated runs - and think about who the receiver is

(Begin of language: English)

Provide different test run results to different roles in the project. A developer will want information from an automated test suite, that a project manager have no interest in. Involve all roles in the project and hear them out on what kind of information they need.

(End of language: English)

Start out small and create the test suites along the way

(Begin of language: English)

In the beginning, create what you need and follow the work flow of the project. Projects change constantly. Don’t waste time, energy, and money on creating something that will never be used or is changed in the last minute. Create the suite along the way, bit by bit.

(End of language: English)

Make your automated setup work hard

(Begin of language: English)

You’ve bought the expensive program, you’ve made the setup, you’ve used a lot of time to create and update the test suites. Use it! Make a suite that can run often AND is valuable to the project. An automated suite that is not running, is a waste of money.

(End of language: English)

Consider that your project and company changes

(Begin of language: English)

Find a program that can grow with your projects and company, and know what you can do – and can’t do – with it. If it isn’t working for you, discard it. There’s no reason to waste your time and money on inadequate tools.

(End of language: English)

Unburden your manual testers

(Begin of language: English)

Try to think of the automated test suite as a helping hand for the testers. Free up their time from tedious, repetitive tasks, so the tester can do what they are best at – testing, not following scripts.

(End of language: English)

Make your suite quick and easy to run

(Begin of language: English)

Setting up and starting automated test runs should be an easy task with some sort of immediate feedback. If your suite is slow or complicated, people won’t bother running them.

(End of language: English)

Keep the suite constantly updated

(Begin of language: English)

The rule is, if your code base changes, then so does your test suite. This is true for ALL code base changes. When bugs are fixed, new features are also added to the system, and that means updating the test suite.

(End of language: English)

And last, but not least...

Don't just accept what some random article on the Internet tells you

(Begin of language: English)

YOU know what you need, so YOU decide whether these Dos and Don’ts apply to your test setup.

Happy automating!

(End of language: English)

Leave a Reply