How to design and test for usability

By: Peter Abrahams



TechSmith are sponsoring a series of roundtables on the subject of usability. The first roundtable took a high level view of the area and investigated the importance of usability, what barriers there are to it being implemented, and how it can be imbedded into the development process;

The second meeting assumed that the importance was understood and the barriers had been removed and concentrated on the question "How to design and test for usability?"

The roundtable was made up of representatives of academia, large enterprise, web development, internet psychology, journalism, usability testing and I chaired it.

There were three main conclusions of the discussions:

Usability design and testing cannot be separated out from the rest of the development cycle. Any discussion must be to ensure that any specific usability needs are built into the development processes. Therefore, any discussion about usability should be to ensure that its specific needs are built into the development cycle.
Usability testing must occur in every step of the development cycle, from ‘paper' testing at concept stage all the way through to monitoring and testing in live production. Testing must be done early enough for the results to be acted upon.
Usability varies by user, by what they want to do, when they want to do it, where they are doing it, maturity of the technology and system, as well as the aims of the site/system owners. It is essential that the system design is based on this variability and not on the comfort zone of the developer.
These conclusion came out of the discussion of four questions:

How to collect usability requirements?
How to design for usability?
What usability testing is needed?
How to run usability testing?
The rest of this article summarizes the detailed discussions of the group on which the above conclusions were based. It is divided into requirements gathering, design and testing. These three areas are inextricably entwined so there is a considerable amount of overlap and cross-referencing between them.

Requirements Gathering

At the highest level the System Usability Scale defines general usability requirements, a system must be:

Effective (the user can successfully achieve their objectives).
Efficient (with the least amount of time and effort).
Satisfactory (the user must be satisfied, and delighted, by the experience).
There are many techniques for extending these general requirements into requirements for a specific project. Very often the new system is a re-launch, replacement or extension of an existing system and reviewing the old system will identify usability requirements for the new.

If the existing system is web based then web analytics can identify areas that users find difficult, where do they give up, where do they loop back within a task, where do they cause an error message to be raised.
Running and recording user tests can then identify the reasons why the user is having difficulties. Analytics help identify a problem; whilst user testing helps diagnose the problem.
Running and recording user test can identify further areas of issue.
Feedback, solicited and unsolicited, will highlight problems.
Users can be interviewed.
Focus groups can be set up and the group dynamics should expose areas of major concern.
If the system is essentially new then other techniques may be used to gather initial requirements. Including ‘paper' prototypes and simple simulation mock-ups. Keeping these simple and rough actually encourages the potential user to criticize and suggest changes as they can see that the cost of change is small both in terms of effort and the designer's pride.

However the requirements are gathered there are pitfalls that need to be avoided:

Not covering a wide enough cross section of user types.
Not understanding cultural differences of the users.
Using language and terminology that confuses or bemuses the user.
Making changes to standard controls; QWERTY maybe awful but it is familiar.
The obvious thing to note is none of the above techniques are unique to collecting usability requirements but are used for all types of requirements gathering. Therefore the cost of collecting usability requirements is marginal when it is done in conjunction with all the other requirement-gathering processes.

Having collected usability requirements early and easily it should be natural for the development team to continue with usability design and testing throughout the project.

There was no time at the round table to discuss the details of these different techniques: how they should be used, when they should be used, what information can be gathered from each and the techniques that can be used to maximize the benefits. This may be the focus of a further round table or article.

Design
The detailed design of the system must ensure that the user can easily do what they need to. The design can be divided into: navigation, process and look and feel.

Navigation
The user should be able to:

Know what can be done.
Navigate to the required place from wherever they are (they may have just entered or may have just completed another task).
To help design the system the developer should:

Try different menu structures.
Check user understanding of terminology.
Provide alternative navigation techniques and potentially provide multiple routes to the same place.
Be consistent across the system and use standard best practice where appropriate (for example on a web page the top left hand corner is often the logo and clicking on it takes the user back to the home page), this should be done even where a theoretically better design might exist, QWERTY layout is so well known it should be used even though other layout should be easier or faster.
Process
It is worth trying different designs for the processes looking at form layout, amount of information processed in each step, the order of the steps, and processing of errors. The process should include the ability to save an incomplete process and return later; this is a useful option to persuade users to sign up so that they do not lose what they have already done. As always the earlier this kind of testing can be done the better and less expensive it will be.

Testing of alternatives can be done in a production system by having dynamic pages that will use different designs for different users. The user reaction can then be monitored and the best design chosen. This is obviously expensive but would be worthwhile before putting a major change into full production.

There is a danger that designers will work within their own experience and comfort zone; watching users struggling with a process should help to pull the designer into the user's experience and comfort zone.

Look and feel
For a user to be satisfied it is not sufficient for the system to be ease and quick to use it has also got to look and feel good. Creative media designers can help to ensure that the system looks right and reflects the user's image of themselves as well as the branding of the supplier. Bringing such designers in should set up some tension between them, the developers, and the usability and accessibility experts. If this tension is managed well it will create a superior design overall.

Besides the system being eminently usable it also must meet the objectives of the owner of the system. The owner may be interested in repeat visits, registered users, new users recommended by existing users, cross selling or larger individual transactions. Generally, usability should assist all of these objectives, however some small compromises may have to be made when compared to the most efficient and effective design.

Designing for accessibility has to be seen as an integral part of the design process. Considering accessibility issues will in general magnify any shortcomings in the design for usability.

A user who has a strong affinity with the product, services and brand of the application may well be more forgiving of the usability of the system; they will persevere through brand loyalty. This effect needs to be recognized and taken into account when reviewing the results of tests.

On the other hand an effective and efficient system can be a major tool in enhancing the brand loyalty and image.

Testing
Testing is carried out to ensure that the system works as intended and meets the objectives of the commissioning organization. In general it is expected that a test will show up some failure or highlight areas that could be improved. It is, therefore, essential that testing is done when it is still possible for the results to have an impact on the project.

Testing is not a one-off event but should occur at all stages of development. Starting with conceptual design, through development, and on into production. The type of testing done at each stage will vary but they must all be designed to maximize the benefit for the cost involved.

Usability and accessibility testing should be done as part of the overall testing of the system. The usability of certain areas of the system will be more critical than others, and the scenarios designed to assess the usability of the system should concentrate on these areas. The important areas are not just those that are used a lot but also areas where it is important that the user has a satisfactory experience; for example when an error is highlighted it is particularly important that the user can fix it easily as if they cannot they are likely to drop out and go to some other system.

To accurately test the usability of the system the tests must be conducted in an environment as close as possible to the environment that the real user will experience. The classical usability labs are not ideal in this regard; the user is away from their normal environment, the one way mirrors can be off-putting and the calm of the lab may not reflect the noise and hubbub of an office or public place. Taking the test to the user in the place and time when it will be used is to be preferred. In reality some compromise solution will be required.

Testing should be viewed by as many involved parties as possible either live or viewing a full recording. Only when a designer starts shouting "Press the red button on the left you dumbo!" will they really understand the importance of good usable design. This does not just apply to the technical designer but also to the creative media designer and the commissioning manager.

When a system has gone into production, testing should continue. Firstly, the usage of the system should be monitored to discover amongst other things, which areas are actually used most, what kind of errors do the users generate, and where do they enter and leave the system. This monitoring should highlight areas that could be improved or requiring more detailed testing.

Production system can also be tested by asking a user to complete a particular scenario. Arranging these tests on a regular basis can identify areas that have been adversely affected by on-going changes to the system.

Conclusion
The design, development and testing of systems are all interconnected and affect one another. Usability has to be an integral part of the development process.

Discussions of individual areas, such as usability testing, are important because they highlight and clarify the issues. However, the implementation must be integrated into the overall development cycle.