Training in the interface: how others have done it before

With a bit more background knowledge from learning science, I have now looked at how this takes shape in interaction design. What kind of research on training and instruction in interfaces already exists, what has already been tested, and what were the results?

To be able to say something about what actually works, I need a measure to judge the solutions against. Endsley (1995) and her concept of situation awareness, the ability to perceive, understand and anticipate what is happening around you, seems to me to give a good answer to this. Whatever design moves you make, this is what you are really building towards, not just how quickly a beginner gets through a task.

With this as the starting point I tried to follow the development from when people first began to look at user experience within HMI, and the history stretches further back than I thought, to several pioneering studies in the 1980s.

One of these studies, from Carroll & Rosson 1987, describes an interesting finding about how users meet onboarding and other training in interfaces. They discovered that new users often skip the manual when they meet new systems, and that this is not due to a lack of motivation, but rather a wish to get going with the task they are there to solve. They call this phenomenon the production bias, a fundamental property of how people learn, which you have to take into account when you design training.

Their own answer to this problem was to build the training into the interaction itself instead. The concept of "training wheels" blocks off the most advanced and error-prone functions early in the learning, so that new users avoid getting lost in functions and error messages they cannot handle yet. A few years earlier, in Carroll & Carrithers (1984), this idea was tested empirically, by removing the parts of a word processor that often triggered error states for beginners. The result was clear: the training group learned the most important things considerably faster, while the control group spent almost a quarter of their time getting out of errors the simplified interface would have avoided.

Forty years after Carroll & Carrithers' study, the same idea turns up again in more modern forms. Forsey et al. (2024) tested a layered interface on a military planning tool, where advanced functions were greyed out for beginners. Here the results were more nuanced: they found no statistically significant difference in the task solving itself, because individual differences between the users overshadowed the effect of the interface. At the same time, the layered interface gave more even performance across users, and was an effective tool for giving a quick introduction to the system. In addition, the authors could conclude that generic interfaces and guidelines across systems support learning and reduce the need for training, which resembles exactly what OpenBridge already lays a foundation for.

I have also looked at a brand new project, Liu & Sra's TaskLens, published this summer. Here they have taken the concept one step further, by using an LLM to automatically build a simplified interface for one specific task. In this way they gather instruction and tools in one place, and give the user access to the functions that the level and the task require. Among beginners this was considerably more effective for learning, but among experienced users the response was divided: several appreciated less clutter, but others felt locked in by the fixed steps and went straight to keyboard shortcuts when the interface did not give them what they needed.

This finding recurs in many of the studies I have looked at: no solution works for everyone, and several point to adaptation to the individual user as something that should be solved. Many have shown that simplification is an effective design move, but it is hard to find a level of simplification that works for everyone.

Perhaps the question is not only how much of the interface to hide from a beginner, but what you are really trying to achieve by hiding it. Here I think Endsley's concept of situation awareness perhaps gives a better measure to design towards than "the right amount of complexity". A training move that gets a beginner quickly through a task, but that does not build this understanding, has perhaps not solved anything at all. That is the question I take with me onwards: not only how you build training into an interface like OpenBridge, but how you know whether that training actually builds this understanding.