17.3. Experiments
Controlled experiments can be used in testing differences between designs, or what makes one design easy to use, or differences between different user groups or task types.
Controlled research environment makes it possible to focus on details of the design.
The researcher can control the situation, the test persons and the tasks.
See example in Box 31.1.
The experiment tries to find out what combinations of pointing - selecting - extending using different mouse buttons are easiest to use in text processing tasks.
Planning experiments
- The purpose of the experiments - what is being changed, what is kept constant, what is measured
- a hypothesis that can be tested using statistical methods
- a statistical test that will be used
Variables
- Independent variables
- the variable that the experimenter manipulates
- Dependent variables
- variables that are dependent on the independent variable
(e.g. D = Time to complete the task, errors, ... I = menu structure, number of icons, ...)
Selecting subjects
- Typical users for the situation
- Control the backgrounds of the persons, maybe using personality tests age, background statistics.
Experimental design
- Independent subject design
- Group of subjects is partitioned in experiment groups randomly
- Matced subject design
- Pairs of subjects are allocated in the two groups and compared
- Repeated measures design
- Subjects appear in both test groups
Independent variables are varied in test groups and the dependent variable is observed.
Critical review of experimental procedure
- User preparation
- Impact of variables
- Structure of the tasks
- Time taken
Critical review of the experimental results
- Size of the effect
- Alternative interpretations
- Consistency between dependent variables
- Generalization of results