A couple decades ago, I got my first exposure to usability testing. This is where we would give folks off the street some cash to come in and do tasks from a list while we recorded their screen. We'd ask them to talk their way through what they were working on so we could understand why they decided to click certain things or type others.
Basically, can normal people use our software? That was the question we were asking.
Task: Turn off the firewall temporarily
Task: Create a network rule to allow Netscape Communicator
Task: Check for product updates
Stuff like this, we'd give a basic task and then we'd sit back and watch through the mirrored-glass (exactly like you'd find in any number of spy or law enforcement movies)
Now I've been working with this program for months and I know it inside and out. I know we're ready to go, anyone can use it and it's a really straight-forward interface so in my mind, this is a formality.
As people started doing the tests, I first began to question the criteria used to pick them. Were these actual adults who could hold normal jobs or did they need constant care? I couldn't understand why they messed up SO MANY THINGS.
"Why doesn't he click the button RIGHT THERE?" I'd ask someone next to me after watching a tester spend a couple minutes hunting for an option.
Later, another new member to the team shook their hands over their head. "I don't get why she doesn't drag the icon into the folder! Has she even USED a computer before? Just drag it!"
"He's in the wrong screen! Why does he think he can make a firewall rule HERE?! I think this is a prank. Nobody could be this thick!" I said to the other team member. They nodded their head
The seasoned developer in the room, would wave at us to quiet down. "Just listen to what they're saying", she told us. "This is why we're doing this."
We finally quieted down when one of our eruptions could be heard through the glass and the person running the test asked us to quiet down. When we listened, we began to actually hear what that developer wanted us to look for. No, it wasn't obvious that they needed to drag something into a folder. Creating the firewall rule in this screen might make sense if they expect this part of the product to work like that other part they tried earlier. That button... I knew that button would take them to the option needed, but that was because I'd played with the product for months. With time, I grew to appreciate these tests because they showed me what we needed to do to make our products better. I started being able to see some of the problems ahead of time, early in design, and that saved us money and schedule. I grew to be better at my job and eventually to really look forward to these sessions.
Reading responses to my story feels like I'm riding an emotional time machine. I'm that younger version of me again. Instead of a mirror-wall, it's the anonymity of the contest and the need to not post (because of the danger of revealing which story is mine) that separates me from the readers. "No", I want to post, "that's not what I meant at all! The character is supposed to be like XXXX, not YYYY. You're wrong!" "That's not it at all, that was on purpose!" I want to tell another poster. After reading a couple responses and yelling at my monitor, I hear that voice from my past again and this time, I really listen.
"Just listen to what they're saying", that developer murmurs to me from the 1990s. I haven't spoken with her in years, but I can still hear that advice like I'm back in Pre- Dot Com Santa Monica. I pause. If I'm going to become a better writer, I can't expect to hand-hold my audience. My story needs to be self-contained, if it needs explanation then I've screwed up. "This", I hear her say in my mind, "is why we're doing this".
