Testing is a vital step on the road to brining something to market, and creating user experience is no different. Geoff Meads explores the different types of testing.

In previous columns we’ve started to delve into User Experience (UX) as a science. If you’ve missed those columns, then UX can be thought of as a measure of the success or failure of a user journey. A user journey being the period from the decision to take an action and the completion of that action

In the smart home world this most often occurs when a user interacts with an installed system. However, UX also covers human and other interactions with, for example, a company on the phone, email, or social media too.

The Importance of Testing

As designers of systems, be they technology or human interactions, we have a responsibility to help our users complete the tasks they need to undertake. This might be as simple as turning on a TV to watch the news, but it could also involve something like making a complaint to a company about customer service. So how do we typically approach the design of that system?

As responsible designers we first research the task, set out a plan of how that task might be completed then build the systems that provide the way for the task to be undertaken. We will then run through the task to prove the system works and voila, we launch it.

But wait, who was it that tested the system? In most cases the test will be run by the designers of the system or other staff close to the process. That being the case, we have a problem – the people undertaking the test already know how the system should work. Many will even know how it works.

These are not ‘users’ in the correct sense in that they already have an expectation as to how the system will work as they built it. Regular users don’t have that knowledge and will have different perceptions and expectations as to how the system might work and this leads to confusion, misunderstanding, frustration and ultimately failure to complete the task.

The Development Testing Cycle

To correctly roll out a new system we can take advantage of a common testing protocol first set out by IBM in the 1950s. The testing protocol falls into a minimum of two stages: Alpha testing and Beta testing.

Alpha testing is exactly as we have described above. A team of developers and designers run the system in as many ways as they can think of to see if it works as expected. At this point, we bring in the original technical specification / operational specification that was generated at the beginning of the design exercise, along with any change notes or updates, to make sure the system performs as specified.

Once Alpha Testing is complete, we move on to Beta testing.

Contrary to popular belief, by the time a beta test is carried out, the system should work pretty much perfectly. The purpose of a Beta test is to use members of the public to test the system or, more usefully perhaps, a set of typical users. An important aspect of the Beta testing group is that they have not been part of the design or development process and that they are typical of the intended user group for the system. For example, if a user interface is being designed and targeted toward people that speak Spanish then a group of English speakers won’t work!

Where we might expect a whole list of bugs to be found (and fixed) during an Alpha test, at the point of Beta testing the number of found bugs should be either small or non-existent.

Designing A Test

Test scheme design can be an amazingly complex undertaking. However, taking a few simple steps in the right direction can really help get most if not all the information that is needed.

First comes ‘User Stories’. While it would be easy to design a test that determines if a TV can be turned on from a remote control it is far more insightful to think first about who might want to turn the TV on and why? The system we design will take those intents into account.

A User Story consists of three parts: a short description of the person who needs to undertake the task, a description of the task itself and the reason the user needs to complete the task. Here’s an example:

“I am a 70-year-old male who would like to turn on the Television to watch the 6 o’clock news”.

What have we learned here? Firstly, we notice the man’s age. At 70 years old, their eyesight and manual dexterity are likely to be limited, even if only slightly, when compared to someone in their 20s or 30s. Next, we note that they are male. This tells us their likely physical size. While this is not a definitive measurement it might help to know this for some tasks.

Finally, they want to watch the news. This means a ten or 20-minute watch time as opposed to a two hour or longer duration for a movie. This tells us that the time from pressing the on button to the news appearing on screen shout be short if not instant. It also tells us they probably don’t want us to be dimming lights and reconfiguring other systems just to watch the TV!

Now imagine the user story is slightly different:

“I am a 20-year-old woman that wants to turn on the TV so I can play computer games.”

Already there are major differences. For example, the gender and age differences suggest a different familiarity with technology and probably (but not definitively) improved manual dexterity and visual acuity.

Note the common requirement – “turn on the TV”. Normally that’s all we think about and, as a result, we miss so much about the real needs and use case of the users.

There’s another short game you can play, the “Why” game. Ask a colleague to state what a button on a control does then ask “why?”. When they answer ask “why?” and so on. It’s a game that comes from the school playground but can reveal an amazing amount about the user’s needs.