Structure evaluation

NOTE: The information in this page is superseded by the availability of online tools such as ChalkMark from Optimal Workshop.

Download 'printer-friendly' PDF version (File size: 14 KB)

Observing users attempt to find items in a program, web site, supermarket or other structured environment reveals considerable variation between individual search patterns. While you can use various techniques (such as card sorting) to develop such structures, the process is very dependent on opinion and educated guesswork.

However, you can evaluate and refine structures using simple paper-based techniques.

Structure evaluation:

Is easy and cheap to conduct
Enables you to evaluate whether users can find items
Is flexible enough to accommodate 'on-the-fly' changes
Does not require a working system, as it can be conducted entirely on paper
Provides solid data you can use to refine the structure.

When is Structure Evaluation appropriate?

Structure evaluation is appropriate whenever you need to test whether people can locate categorized items - for example, within a web site.

Structure evaluation is best conducted at the early stages of design, when changes are relatively easy and inexpensive.

Conducting Structure Evaluation

Assume we want to evaluate the following simple web site structure. (Note that each item is numbered.)

About us	Support
24 Corporate profile 15 Investor information	12 Technical support 3 Downloads 23 Software Drivers
Contact	Products
4 Locations 18 Phone numbers 35 Email	1 Software 53 Hardware 11 Services

We can represent the high-level structure of this site by laying four sheets of paper on a table:

Image showing 4 sheets of paper, each representing a category from the previous table

On the reverse of each sheet is the list of items it contains. For example, the reverse of the 'Support' sheet contains the numbers '12', '3', and '23' (since these are the items that belong in that category).

A participant is presented with an index card representing an item:

Index card, showing title of an item ('downloads') and a number ('23')

The participant attempts to locate the item in the structure. The reason for using a number is that this reduces the ability of participants to remember items already seen. Each item is presented in turn (in a random sequence) until the participant has attempted to locate each one.

All attempts to locate items are logged. Allow participants a pre-defined number of attempts to locate each item. Remember that like usability testing, this sort of evaluation can be stressful for participants.

Once all participants have completed the exercise, enter the data in a spreadsheet, and examine the groupings.

You can use cluster analysis to get a pictorial representation of the resultant groupings.

Pay attention to items that are particularly difficult to locate, and consider whether they should be renamed or relocated.

Who should participate?

Involve as broad a range of participants as possible from within the intended user group. Avoid using participants who within the development team or client organization, as they will have an unrealistic familiarity with terminology.

If possible, use at least 8 participants to get a spread of results. Be aware that the more participants you have, the more data you will need to analyze.

Variations

Since evaluations are quick, consider combining them with other forms of usability activities to maximize the benefit of having users present.