The Importance of User Testing

My design experience has taught me the value of user testing and feedback. I’ve observed projects that dismissed user testing, and the inevitable wave of negative reviews, or late-schedule drastic reworking, that resulted. I’ve focused much of my education on the methods of user testing that ensure a “user-centered” design that works for the real life user, not just the persona in the designer’s head.

In the fall of 2024 I worked with a team of 5 fellow students — Adam Pywell, Irene Lau, Jonathan Supangat, Justin Shu, and Nanop Yansomboon — to analyze the tutorial for the online tabletop gaming platform Roll20.

Through user testing that involved questionnaires and task performance analysis, we were able to diagnose serious usability problems. One major issue being that most users failed to recognize the tutorial as interactive; instead, they quickly clicked through a series of text-heavy didactic windows without actually learning how the system worked. This resulted in very poor performance in the tasks designed to test their understanding following the tutorial.

Responsive Design

The team then brainstormed design interventions that might result in greater user interaction with the tools and systems presented during the tutorial. My idea was to add a “Let Me Try!” button to the tutorial windows alongside the existing “Back” and “Next” buttons.

Let Me try!

It was our theory that most users only recognized 2 options during the tutorial — “Back” and “Next.” This discouraged users from attempting interaction with anything other than those 2 buttons during the tutorial. The “Let Me Try!” button was a simple way to offer greater interaction.

Results

Testing of the prototyped “Let Me Try!” button produced a massive jump in user interaction during the tutorial. The mean time spent interacting with the tutorial jumped from just over 4 minutes to over 23 minutes, and total user clicks went from a mean of 33.8 to 196.5!

Post-test reports by users also showed a significant jump in rating of helpfulness and pleasantness of the tutorial.

In statistics lingo, we had rejected the null hypothesis. We had also shown the massive impact that can be made, to both user performance and satisfaction, by the smallest change to an interface.