The following books are books recommended by Gerry Gaffney.
Beyond the Usability Lab: Conducting Large-Scale Online User Experience Studies, by Bill Albert, Tom Tullis, Donna Tedesco
This review originally appeared in User Experience magazine in 2011
In 2008, User Experience Magazine devoted an issue to remote usability testing
guest edited by Tomer Sharon. As managing editor at the time, I confess
to having had a rather skeptical view of the value of remote testing.
One of the articles that made me re-think my attitude was a case study
comparing the relative usability of two sets of information about the
Apollo program, one provided by NASA, and one by Wikipedia. That
case study was written by Tom Tullis, one of the authors of Beyond the
Usability Lab, which takes the themes introduced in that issue and explores them in detail.
Traditional usability testing has been focused on small sets of users, and
the book does not take issue with this approach. Indeed Bill Albert said to
me in an interview, “…if the goal is really just to identify usability issues,
I think I’d fall in line with a lot of other people, saying six to eight users is
plenty to identify the significant usability issues with a particular design.”
What the 2010 book concerns is the fact that web technology “enables
us to move beyond the lab and efficiently conduct user experience studies
with a much larger sample of users.”
The book begins with an introduction describing what the authors
mean by “online usability studies,” including a description of when
such studies are appropriate, what one can expect to achieve, and the
strengths and limitations. Remote studies are good for comparing
designs, for collecting detailed and scalable usability metrics, and for
exploring design issues in the users’ own environment, with all its
attendant complexities. On the other hand, there are many instances
(such as identifying the major usability issues with early prototypes)
when other methods are preferred.
The book is well-structured for the practitioner. After the introduction,
the following three chapters explore planning, designing, piloting, and
launching the study. The hands-on approach is reminiscent of Rubin’s
(and now Chisnell’s) classic Handbook of Usability Testing, in that it
contains sufficient detail to enable a practitioner to engage the method
with a degree of confidence.
Chapter 7 provides good in-depth analysis of specific tools (Loop11,
RelevantView, UserZoom, and WebEffective) that can be used to conduct
remote studies, as well as advice on choosing the appropriate tool for
your own study. Chapter 8 discusses discount methods, including “totally
homegrown techniques for the adventurous.”
Chapter 9 presents seven case studies of remote research conducted
with between 24-300 users with a range of tools.
Throughout the book, specific examples illustrate concepts and
methods. The authors provide detailed instructions for using Microsoft
Excel to calculate appropriate averages and confidence intervals. There
is also advice on dealing with data gathered from open-ended questions
(when simple numerical analysis is not adequate).
The authors describe how to identify and deal with data from “flatliners”—
participants who complete studies as quickly as possible to
obtain the associated incentive.
It’s a real pleasure to encounter a book that not only takes the reader
on a journey through the rich possibilities of technique, but does so in a
manner that is clear, readable, and accessible. I was particularly pleased
with the simple explanations of statistical techniques, which are so often
presented as incomprehensible.
If you’re interested in any of the following questions, you can look to
this book for practical and effective answers:
- Should you conduct a between-subjects or within-subject study?
- What variables do you need to consider?
- How can you deal with outliers?
- How can you calculate and display confidence intervals?
The book does not shy away from the difficulties involved in conducting
remote research. For example, if you want click-stream data, it may be
necessary to have participants install or allow a plug-in, which may mean
you can’t test with so-called novice users.
If I were to complain, it would be about the need for a chapter
specifically on conducting studies on mobile devices—an area that is ripe
for a similarly detailed “how-to” guide.
Whether you’ve conducted remote studies in the past and want to
extend your capability and knowledge, or you are a complete newcomer,
this excellent book is a necessary companion on your journey from the lab
into the world outside. You will refer to it often, and it will alert you to
opportunities and dangers. What more could you ask of a book?
Practical Speech User Interface Design, by James R. Lewis
This review originally appeared in User Experience magazine in 2011
James R. Lewis’ Practical Speech User Interface Design is comprehensive,
accessible, practical, and fascinating. As an IBM human factors
engineer for some thirty years, Lewis brings a depth of practical experience
to bear in this book. He has also added the breadth of the current
state of research in the field.
As he states in the closing pages, some of the research he reviewed
“confirmed my current design practices, but more importantly, other
research has led me to make some changes in my design strategies.” It
is this combination of openness and expertise that makes the book such
an asset for anyone interested in speech user interfaces—and what UX
practitioner isn’t? I’ve had the pleasure of working on a small number
of speech systems, and only regret that I didn’t have this book prior to
embarking on them.
Chapter 2 introduces speech technologies. Lewis describes two types
of language models in current use—finite state grammars (in which
all legal words and phrases are fully specified) and statistical language
models (in which users may speak the words in any order). This chapter
also describes methods of speech production and discusses formant textto-
speech (think Stephen Hawking) and concatenative text-to-speech.
While intelligibility of synthetic voices was once problematic, current
systems are generally easy to understand. However, the production
of “convincingly natural and appealing” synthetic voices has been a
challenge, and Lewis points out that businesses are reluctant to risk their
brand images. Accordingly, most designs use recorded speech.
Lewis also discusses the use of speech biometrics (voiceprinting).
However, he suggests that current accuracy means that it can only
be used in low-security applications or when combined with other
Chapter 3 is a fascinating discussion of human speech and its
implications for design. It includes material on phonology (the study of
the basic sounds of language), coarticulation (the phenomenon whereby
we run words into each other), and prosody (intonational patterns). A
section on discourse considers the patterns involved in our everyday
conversations, and includes material on grammaticality, discourse markers
(which signal conversational intents), timing and turn taking, and social
considerations in conversation. Here, as throughout the book, the material
is clearly presented, supported by critically considered research, and
focused on being of practical use to the user interface designer.
This focus on the practical also means that the author largely avoids
prescriptive statements; rather, he recognizes that all decisions must be
subject to the constraints faced by the business and designer. Where
prescriptiveness is possible, however, Lewis provides unambiguous
advice by clearly identifying appropriate durations of wait times,
silences, and pauses.
Chapter 4 considers self-service technologies, their advantages and
disadvantages, and the propensity or willingness of people to use them
and the factors affecting this willingness.
In the meaty Chapter 6, “Speech User Interface Development
Methodology,” the steps listed are largely familiar (requirements,
design, develop, test, deploy, tune). While user needs analysis is dear to
the heart of any UX practitioner, Lewis describes the specifics of doing
the analysis for speech. For example, he suggests that (in a project
lacking countermanding data), “it is reasonable to design in accordance
with the capabilities of older adults.”
Lewis covers creating detailed dialog specifications, prototyping,
development, and testing. This chapter will serve a UX practitioner
transitioning from GUI to SUI design particularly well, as it enables the
application of existing knowledge to the new domain.
There is an interesting discussion on the use of personas. Lewis
states that while personas can be useful, they should not consume
a large part of the design effort, which will be better rewarded by
Chapter 8 gets into the gritty detail of how to script introductions,
whether to tell people to “listen carefully” because of changed menus
(don’t), how to provide help, and a discussion of appropriate menu
lengths. Lewis states that the common advice to limit menu length is a
mistake (a misguideline to use his term), and that broad menus are more
effective than shorter menus requiring greater depth. This might be seen
as controversial (my eyebrows certainly went up), but, as always, the
author quotes extensively and critically from the available research and
his own work.
Each chapter in the book concludes with a summary, and a reader who
does not need the details can simply read the summary to gain a fair grasp
of the content of the earlier pages.
If you have even the vaguest interest in design of speech user interfaces,
and whether you’re a student or a seasoned practitioner, read this book.
Note: You can read or listen to an interview with Jim Lewis on the User Experience podcast.
Donna Spencer’s “Card Sorting”
Whether you’re a complete newcomer to card sorting, or an experienced practitioner, Card Sorting: Designing Usable Categories will be of interest and of use.
For the newcomer, it provides step-by-step guidance and straightforward advice on how, when and indeed whether to carry out card-sorting activities.
The book begins with an exploration of information categorization. This is written in a very accessible fashion, although it doesn’t shy away from the occasional deeper foray into topics such as classical views of classification.
Most of the remainder of the book is a hands-on guide to planning and conducting card-sort activities. While the author does discuss the use of software tools, she is a firm advocate of the use of physical cards.
The book will also prompt experienced practitioners to question and re-evaluate the practices that they may feel they know thoroughly. For example, the statement that individuals can handle more cards than groups took me by surprise, although on reflection it confirms my own experiences.
At times I found the advice to be overly-prescriptive. For example, Spencer instructs the reader not to mix function and content in a card sort. While the rationale is sound, in my opinion there are plenty of instances when it is perfectly reasonable to mix these elements to good effect.
The book is very much focused on providing practical advice for practitioners. If you want an academic discussion, you must look elsewhere.
The use of case-studies and examples contributes significantly to the readability of this book. Spencer’s frank admissions of instances when she (and others) “got it wrong” were refreshing and also provided a learning method that is almost as good as making one’s own mistakes.
The author also offers a key piece of advice towards the end of the book: “Don’t rely on a technique to do your thinking.” Sensible advice, indeed.
Bill Buxton’s “Sketching User Experiences”
Highly recommended, “Sketching User Experiences” is thorough and thought-provoking, and about much more than sketching.
Buxton discusses the process of design, and the factors that can lead to success. Essentially, he claims that a design process is simply absent, and that the bulk of our industry is organised around two myths: “that we know what we want at the start of a project, and that we know enough to start building it”. This is reminiscent of Brooks’ “Mythical Man-Month”, and indeed this is included in the extensive bibliography.
Buxton ranges wide in his use of examples, examining aspects of Apple, Dell, Microsoft, and Trek Mountain Bikes. He uses the advent of the mountain bike to discuss how innovation can revolutionise an apparently stable or stagnant industry. It is fundamental to his belief system, he says, that “there is always a mountain bike” (strange echoes of Flann O’Brien’s “The Third Policeman” here). He also quotes a wide range of academic research, and this thoroughness makes the book even more compelling.
The book is well and liberally illustrated. The range of products discussed ranges from juicers to ski equipment, but the focus is very clearly on the design of interaction and the sketching techniques that can help examine and design interactions.
Buxton hits his stride when he pins down precisely what he means by a “sketch”, and identifies it as something which has a specific set of criteria. It must be quick, disposable, plentiful, use a clear vocabulary and distinct gesture, have minimal detail, an appropriate level of refinement, should suggest and explore rather than confirm, and should be intentionally ambiguous. Armed with this definition, he proceeds to broaden our thinking – certainly my thinking – of what might constitute a sketch. He includes hand-drawn conceptualisations, but also sketches consisting of stories, performance, 3D objects, video, animation and mock-ups of various sorts – provided they meet all the criteria. Perhaps unavoidably, the precise but broad definition occasionally seems to allow arbitrary inclusion or exclusion, although he is at pains to distinguish between a sketch and a prototype.
The book has a narrative, and therefore needs to be read in sequence. It’s not one that you can easily dip into (although I expect to refer back to it frequently); the thesis is built progressively. It’s interspersed, however, with highlighted comments or exhortations, and I found myself fairly active with the highlighter (always a good sign).
A constant theme is the need for those of us in “interaction design” to learn from the traditions and practices of other design disciplines, and Buxton describes the use of corkboards (“awareness servers”), design crits, and more. “All this is the norm,” he writes, “and hardly even needs to be said in a traditional design studio. But what we are talking about in this book is not the traditional design studio, but a studio populated by people who come from other traditions than, for example, graphic or industrial design.”
Every book should change your life. After reading Buxton’s, at the very least, I’ll henceforth abandon the term “low-fidelity prototype” since, as he points out, “when… properly used, they are not low fidelity; rather, they are at exactly the right fidelity for their purpose.”
A thoughtful and reflective reading of this book by anyone involved in design, product engineering or usability will of necessity change the way you think about your work, its history and tradition, the techniques you use, and their application.
Kim Vicente’s ‘The Human Factor’
This excellent book reminds those of us who spend most of our time working within the relatively
narrow confines of the user interface that there are many layers to the requirement to build
technologies that enhance the human condition.
Vicente begins by categorising the human factor into five levels – physical,
psychological, team, organizational and political. He then discusses each level
in turn, beginning with the physical.
Vicente’s view of what constitutes design is very broad – reaching right up to the
level of governmental policy. ‘Policy aims’, he states, ‘are the mother of all system design decisions’.
There is a fascinating discussion of the way in which the design and implementation
of a no-blame reporting system has improved airline safety dramatically, as well as
a disturbing explanation of why a ‘shame-and-blame’ system is preventing the same
sort of reduction of accidental death in the health care industry.
The book contains many case studies and examples – from relatively simple user
interface examples to control rooms, health care systems and politics.
Vicente argues for a revolution in the way we think about technology, and for what
he terms a ‘Human-tech’ approach to design.
This is not a ‘how-to’ book. Instead, it offers a profound consideration of the
philosophical underpinnings of human-centred design, and suggests ways forward that,
if chosen, have the potential to produce immense benefits for our future.
Highly readable, and highly recommended, this is not just a book for
usability practitioners, but for anyone with an interest in society at large.
‘Observing the User Experience’
In Observing the User Experience: A Practitioner’s Guide to User Research,
Mike Kuniavsky describes in detail all the major techniques for understanding users and their needs; contextual enquiry, focus groups, surveys, questionnaires, diaires, usability tests, and more, in sufficient detail to enable practitioners to undertake them in an organised and informed fashion.
The no-nonsense approach includes guidelines, tables for estimating costs, and plenty of detailed practial advice. Simply written, well-structured and highly informative. Don’t leave home without it.
‘The Cognitive Style of PowerPoint’
Edward Tufte uses a simply bound booklet format to present an essay on the shortcomings
of PowerPoint, which he claims tends to impoverish rather than enhance the effectiveness
and value of presentations. He is also highly critical of those who thoughtlessly use
PowerPoint as a substitute for adequate preparation and presentation.
He bemoans the low resolution of PowerPoint presentations when compared with paper hand-outs, and the inappropriateness to purpose of the supplied templates.
As usual, Tufte supports his arguments well with examples; as usual, he is uncompromising in his comments.
Tufte does not consider PowerPoint in other contexts (such as, for example, producing
wire-frames for web or software development).
I found this essay to be provocative in many ways. It certainly forced me to re-evaluate my own use of the program for presentations and lectures.
Are you using PowerPoint to enrich your presentations, or are you diminishing
your arguments, your credibility, and the data you rely on, by using a medium which
is simply unsuited to what you are trying to communicate?
At US $7 (in late ’03), this is a bargain buy. You can purchase ‘The Cognitive Style of PowerPoint’ from Tufte’s website.
‘The Elements of User Experience: User-Centered Design for the Web’
In The Elements of User Experience: User-Centered Design for the Web,
Jesse James Garrett presents a model with five planes; strategy, scope, structure,
skeleton and surface.
After introducing the model, he explains the relationships and dependencies between
the planes, and then discusses each plane in some detail.
One particular advantage of this model is that it applies to both interactive and
publishing models (‘web as software’ and ‘web as hypertext system’), and can be used
when developing websites that contain elements of both.
The book itself is a model of clarity, elegance and usefulness.
For anyone involved in any aspect of web design (or indeed, interaction design in
general), I recommend this book highly. You can read it cover-to-cover in a few hours.
‘Information Architecture: Blueprints for the Web’
Christina Wodtke covers more ground in Information Architecture: Blueprints for the Web than the title might lead one to believe.
It covers (to greater or lesser degrees) principles of usability, user needs analysis, categorisation of information, metadata, the use of storyboards, personas and scenarios, and user interface elements.
There are plenty of examples provided to illustrate and clarify. The lack of colour reduces their efficacy somewhat, but also no doubt reduces the price of the book. Wodtke also provides many interesting and useful references to websites and other resources.
The book will be of interest and use to both newcomers and students and to practitioners.
The section on documentation will be useful to many readers, containing information on drawing and sketching, storyboarding, diagramming and the production of necessary project documentation.
Although I read the book cover-to-cover in a single day, it would be better to read it piece-meal with highlighter in hand, to allow sufficient time and attention to consider and absorb the content, and to make the book more useful for future referencing.
‘Contextual Design: A Customer-Centered Approach to Systems Designs’
Beyer and Holtzblatt’s
‘Contextual Design: A Customer-Centered Approach to Systems Designs’ is an excellent
book that describes techniques for gathering data from customers and applying it to the
design process. It provides good advice on a variety of models for conducting user
interviews, including the ‘master-apprentice’ model. The book describes in detail a
number of techniques for modelling and presenting the data you collect. It also describes
User Environment Design, a techique I have found useful for approaching the problem of
translating data into interface designs.
‘Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests’
A ‘must have’ book is Jakob Nielsen’s
‘Usability Engineering’. This is an excellent introduction to the philosophy and
practice of usability.
‘The Inmates Are Running the Asylum : Why High Tech Products Drive Us Crazy and How To
Restore The Sanity’
For a provocative and entertaining view of interaction design, check out Alan Cooper’s
‘The Inmates Are Running the Asylum : Why High Tech Products Drive Us Crazy and How To
Restore The Sanity’. Cooper has some harsh words to say on the subject of usability.
He also has a very interesting and rather unflattering perspective on the psychology
‘The New Drawing on the Right Side of the Brain’
Although not a ‘usability’ book,
‘The New Drawing on the Right Side of the Brain’ by Betty Edwards is a great
introduction to drawing. Edwards states that if you can write your name you can draw; she
then proceeds to prove this with a series of exercises.