Test frequently and catch/fix any usability issues before launch.
With the rise of mobile usage, Avvo needed to design and develop a mobile version of the site. The
usability schedule kept the design and development moving forward. We did smaller tests more
frequently, fixed bugs and quickly re-tested the fix. This form of iterate/test was easier to digest
and didn't jeopardize the launch date since we found things early.
Coordinate all aspects of testing with the product team, usability consultant and
At the time Avvo didn't have an in-house user researcher so we needed to hire an out-side consultant
or agency. I reached out to a few usability consultants in Seattle and had them come in and meet
with the product team. The product team decided to do as much in-house as we could and hire a
consultant to do the moderating.
Coordinated all the efforts of the tests.
- Drafted the tasks and reviewed with product team
- Created the schedule
- Assisted with recruiting requirements
- Hardware set up and configuration
- Pilot tests
- Participant compensation
- Consent forms, NDA agreements
- Observation set up
- Facilitate note taking during observation
- Facilitate triage of bug fixes
- File bugs
I suggested we do smaller tests (5 participants) more frequently instead of one large elaborate
test. Going by Jakob Neilson's iterative design suggestion I scheduled 3 different test
over a 6 week period. The smaller test helped keep the design and development moving forward and we
caught usability issues early. It was also a lot easier to manage the test in smaller chunks.
The hardware set up was a challenge - In 2011 there wasn't a lot of resources for doing mobile
usability. I had to figure out how to capture the device while letting the participate hold the
phone. We considered fabricating a case that would mount the webcam to the device. However that made
it very awkward for the participant. Through many searches I discovered a document camera that was
freestanding. As long as the participant held the device under the camera it would capture their
hand gestures and the screen. So I ended up using two IPEVO document
cameras. One that recorded the participants hand/phone and the other recorded there facial
expressions. We used Morea to manage all the recordings and observation room.
I set up the observation room on the other side of the office so the participant wouldn't over hear
the team. During the test members of the product team would take notes. At the end of the session we
would white board our findings. At the end of the day we would compare notes from all participants
to see if there was a pattern of usability issues. PM would decide what was a priority to fix before
the next usability tests.