NOTE: The information in this page is superseded by the availability of online tools such as ChalkMark from Optimal Workshop.
Download 'printer-friendly' PDF version (File size: 14 KB)
Observing users attempt to find items in a program, web site, supermarket or other structured environment reveals considerable variation between individual search patterns. While you can use various techniques (such as card sorting) to develop such structures, the process is very dependent on opinion and educated guesswork.
However, you can evaluate and refine structures using simple paper-based techniques.
Structure evaluation is appropriate whenever you need to test whether people can locate categorized items - for example, within a web site.
Structure evaluation is best conducted at the early stages of design, when changes are relatively easy and inexpensive.
Assume we want to evaluate the following simple web site structure. (Note that each item is numbered.)
|24 Corporate profile 15 Investor information||12 Technical support 3 Downloads 23 Software Drivers|
|4 Locations 18 Phone numbers 35 Email||1 Software 53 Hardware 11 Services|
We can represent the high-level structure of this site by laying four sheets of paper on a table:
On the reverse of each sheet is the list of items it contains. For example, the reverse of the 'Support' sheet contains the numbers '12', '3', and '23' (since these are the items that belong in that category).
A participant is presented with an index card representing an item:
The participant attempts to locate the item in the structure. The reason for using a number is that this reduces the ability of participants to remember items already seen. Each item is presented in turn (in a random sequence) until the participant has attempted to locate each one.
All attempts to locate items are logged. Allow participants a pre-defined number of attempts to locate each item. Remember that like usability testing, this sort of evaluation can be stressful for participants.
Once all participants have completed the exercise, enter the data in a spreadsheet, and examine the groupings.
You can use cluster analysis to get a pictorial representation of the resultant groupings.
Pay attention to items that are particularly difficult to locate, and consider whether they should be renamed or relocated.
Involve as broad a range of participants as possible from within the intended user group. Avoid using participants who within the development team or client organization, as they will have an unrealistic familiarity with terminology.
If possible, use at least 8 participants to get a spread of results. Be aware that the more participants you have, the more data you will need to analyze.
Since evaluations are quick, consider combining them with other forms of usability activities to maximize the benefit of having users present.