Yesterday, the Danish part of House of Test met up and discussed life and testing over tiramisu (Which in my opinion is an ideal starting point for discussing anything). We ended up talking about mentoring and coaching, and how one becomes a better tester. That led us to discuss a statement that you sometimes meet when faced with a system that you simply do not understand: “But I don’t know how to test it” or “I don’t know the system, so I can’t test if it work correctly”

The value in meeting a system for the first time

Last month I did a one-week project where I put an app through usability testing. The company that hired me was very careful not to tell me a lot about the app and their system, since they wanted me to see it for the first time. That was a very wise decision on their part. While I performed the first testing, and got to know the app, I was struck with the value of this very first meeting with the system. There’s a very important and sometimes overlooked lesson in looking at a system with new and fresh eyes. It’s a fleeting moment, but the very first time you set your eyes upon a system, is a test in itself. So what can you do, even if you have very little idea about how the system you’re going to test, will work?

1. We can all make assumptions

We assume things all the time. We take in clues and symbols, and hold that up against our own experiences and frame of reference. From that, we assume things. About the world, about people, about things, about systems. When faced with a system you’ve never met before, use your assumptions as a starting point for testing. If it’s a well designed system, you will know instinctively what the system is about. There will be clues and helpful text that can guide you through the system. If you don’t understand it, it’s not because you’re stupid. It’s because it’s badly designed. There’s a lot of badly designed systems out there. So write down those assumptions you have before the first meeting, and allow them to change as you grow wiser on the system.

2. Prepare for the first meeting and set some test missions

Reserve some time for the first test, and make sure you have ways to document it (More about that in point 3). Be comfortable and prepare yourself for meeting the system for the first time. Be aware of your assumptions about the system. Get into the mindset that you are meeting the system for the very first time, and that it probably will be confusing. It’s perfectly okay. To help you on your way make one or several small test missions that you presume the system’s user would have. It could be anything from “I want to sign up and log in” or “I can search for a trip to Russia”, to “In the overview I can see relevant information on any customer who sent in a health declaration”. You might not know if, what you think is relevant information, is what is actually relevant. But it’s a starting point, and you might just notice that some data is missing from the list.

3. Note down everything

Everything.Turn on your recorder in your phone, and talk your way through the first meeting and your test missions.If you can’t fulfill your test Note down every single thought you get, such as:

• Assumptions about what and who the system is intended for
• Assumptions about how the system is supposed to work
• Assumptions about how the system actually work, and how these assumptions slowly change as learn more about it
• Any fleeting thought that triggers a “this is weird/annoying” response in you

It takes some time to summon the courage to note down the “stupid” questions. Questions where you find yourself thinking “I probably don’t understand this because I don’t know how the system works”. Note those down as well. Even if you feel like it’s a silly thought. If you have it, it’s valid, and others will probably have that thought as well. You can always remove thoughts from your notes later, but you cannot remember all your thoughts that you didn’t note down on the first meeting. It would be such a shame if that weird thought that actually held a crucial detail, was discarded.

First meeting testing – An addition to your test tool library

I must point out that I don’t think this should be the only way to test a system. There are some systems that require expert knowledge, and are only used by expert users. I’m working with large and complex financial systems, where I need to learn a lot of things before I can fully understand what needs testing and how. I rush through examples, guides, business rules and requirements to fully grasp when a health declaration can and should match a request for life insurance. I have to understand 5 different programs. Not know how to use 5 different programs, understand what takes place behind the user interface of each one of them. If I didn’t know all the business rules, and I didn’t do any other testing than the above mentioned method, the systems would fail miserably. I will however say that when a new program was developed in my project, and I was asked to test it, I did a “first meeting” test. I did that because I knew that it was a completely new program, that none of the experts had any experience with. That led to several bugs being found, followed by a lot of adjustments in the user interface. And that in turn led to a happier end-user. So there might just be room for a first meeting test in your complicated project as well.