171. Observing and monitoring performance

Direct observation

Find out how users work or use their equipment
Quick observations
Plan what you want to find - checklist
Hawthorne effect - users are aware of the observer
Respect the users

Indirect observation : Video recording

Observe the screen, keyboard, other tools and the users actions
Need probably two cameras
Plan what to observe
Plan how to analyse
Possibly synchronize with program log

Analysing video data

Task-based analysis
Performance based analysis

Frequency of correct task completion
task timing
use of commands
frequency of errors
cognitive activities

Very time consuming (1 hour of tape takes 5 hours to analyse/log)

Use only if in depth interpretative analysis is needed.

Verbal protocols

Record the user thinking aloud.
Synchronize with screen images (video or computer log) or with observers verbal log
Observations of users feelings, problems, problem solving situations

Software logging

All actions of the user are logged by the computer. Later the analyst can play back the actions and analyse them.

Time stamped key-presses - actions logged
Interface logging - screen images "videotaped"

In typical settings a number of techniques are used - video-observation - action logging - thinking aloud.