For several years, usability has no longer required justification in most quarters:
"Usability has become a competitive necessity for the . . . success of
software" [Butler, 1996]. Because of the growing awareness of its importance,
organizations have been expending resources for "doing usability"
- building enviable usability laboratories, buying usability equipment, training
developers in usability engineering methods [Hix & Hartson, 1993], and conducting
usability testing. However, those same organizations are often not achieving
acceptable returns on this investment "downstream" in the usability
process (after testing), where there are few techniques and almost no tools
to support problem extraction, analysis, documentation, reporting, and data
management of usability problems found during user-based testing.
Lack of a conceptual framework, standard usability vocabulary, and support tools
to organize and guide usability problem reporting and subsequent usability redesign
activities causes information losses and poor communication of usability data
within the iterative user interface development cycle. When usability problem
communication relies too heavily on human memory and word of mouth, developers
can only try to interpret and reconstruct the missing usability information.
Finally, usability engineering research that has been done has been slow to
reach practitioners and get into practice.
Our proposed development of usability engineering methods and support tools
will provide return on investment in usability engineering activities downstream
from usability testing by filling the need for:
Having analyzed hundreds of usability problem descriptions collected from commercial, government, and military organizations, we repeatedly found the typical result of costly usability evaluations to be little more than a laundry list of raw usability problems with descriptions of inconsistent quality, often vague and incomplete. The exigencies of real-time data collection make it very difficult to do much more than that during a usability evaluation session. After the session various, ad hoc attempts at classification may sometimes be performed and some forms and checklists, for example, exist for documenting the data. However, there is no framework to guide and structure this activity, crucial to preventing information losses between observation in the usability lab and reporting of the usability problems observed. Our new usability problem classification tool provides a structured method for classifying and documenting usability data to produce high-quality usability problem descriptions.
Ad hoc UE analysis methods depend on the expertise and experience of the individual practitioner. The UAF-based tools are desigend to present a structured methods, offering more hope for consistently good results. In many ways, usability problem classification is a kind of diagnosis. A practitioner must know exactly what the problem is, in order to fix it properly. Analysis via the Usability Problem Classifier determine the problem in terms of effects of the user, problem type, and causes of the problem within the interaction design. Problems of the same type can appear to be different in the surface observables and problems that look the same on the surface can be quite different in the underlying problem type and causes. Practitioners can think they are discussing (and disagreeing about) a problem without knowing they are talking about two different problems.
Finally, there are no practical usability data managment tools to support real-world usability practice in a project environment.