In the field of usability, debates about number of users, the use of statistics, etc. in the abstract are pointless and even counter-productive. I propose that the answers depend on the research questions and business objectives of each project and thus cannot be discussed in absolute terms. Sometimes usability testing is done with an implicit or explicit hypothesis in mind. At other times the purpose of testing is to guide iterative design. These two approaches call for different study designs and treatment of data.
Control systems theory is very applicable to the topic of usability to highlight and frame the value of iterative usability testing in the design lifecycle. Within this new metaphor, iterative testing is a form of feedback which is most effective and resource-efficient if done as often as practically possible with project resources and timelines in mind.
In the basic control system feedback loop, the “input function” is the sensing of the current state. That perception is compared against a point of reference through a mechanism called “comparator.” If a discrepancy is perceived between the present state and the desired (or reference) state a behavior is performed, which is the “output function.” The goal of the output function is to reduce the discrepancy. The output has an impact on the system’s environment (i.e. anything external to the system). Such an impact creates a change in the present condition, leading to a different perception, which in turn is once again compared with the reference value (Carver & Scheier, 1982, p.11).
CST is a useful theoretical framework for usability testing for several reasons. Firstly, usability testing is as an integral part of the User Centered Design (UCD) process which is a complex system driven by goals. Secondly, UCD and usability goals are hierarchically organized. Thirdly, iterative usability testing (as opposed to benchmark testing) can be viewed as a form of continuous feedback that guides design.
- Do not debate the appropriateness of specific user research methods in the abstract.
- Before selecting a research method, always clarify the research questions and business objectives of each project and get team buy in.
- Consider iterative usability testing a form of feedback on the progress towards specific design and business goals.
- Start doing iterative testing as early as possible in the design lifecycle.
- Conduct iterative testing as often as practically possible with project resources and timelines in mind.
Link to full article: Iterative Usability As Continuous Feedback