Usability Tests – Methodology

Nielsen, J. & Norman, D. (2000). E-Commerce User Experience: Methodology.

The methodology section describes the testing parameters for the usability tests on small and large e-commerce sites. By selecting sites that didn’t require involve special skills or products and by eliminating users who didn’t know enough or who knew too much, the tests attempt to achieve usability for average consumers. While the concept of task independence (completion of a task foes not depend on successful completion of an earlier task) makes sense, this requirement can create difficulties when designing multi-step workflows; the solution is to create larger self-contained tasks.

While the methodology section discusses generalities of user selection, it also offers practical and specific suggestions:

• Define tasks in increasing order of specificity
  • First impressions
  • Specific task
  • Open-ended task
  • Customer service (particularly appropriate for e-commerce)
• Create tasks to avoid bias
  • Describe goals but not steps
  • Avoid terms from the website
  • Put each task on a separate page
• Use a pilot to refine the test
• Employ an interactive think-aloud protocol without providing assistance (unless the user gets stuck – see next suggestion)
• Provide technical assistance for users who become stuck (sites are being evaluated, not users)
• Type continuously in the data collection mode (less intrusive than intermittent typing)
• Recognize that users may interpret post-test survey questions very differently (and often less critically) than the test indicated
• Use a comprehensive informed consent agreement