Recording Usability Tesing with GoToMeeting

GoToMeeting, a great tool for remote meetings, is also incredibly useful for moderated usability testing. Using GoToMeeting can help you eliminate the number of observers while still allowing stakeholders to view testing. In addition, the recorded files are small, and the setup is simple.

In recent moderated usability tests, we have started using GoToMeeting to share our sessions. It’s particularly handy for setting up a separate observation room without needing a large lab. We set up a second computer as a guest in a remote viewing room in the GoToMeeting tool, which allows all our observers to come and watch. (We’re careful to mute the observing computer.) This method keeps the number of people watching a session, live and in person, to a minimum. This also keeps the test participant from becoming distracted or intimidated by the large number of observers.

The second and most useful feature is GoToMeeting’s recording option. We can use GoToMeeting to record the screen of the laptop as well as the voices in the room. The file size and quality are both pretty reasonable. For both desktop and mobile testing we can set up a small camera window in the lower corner of the screen that shows the participant’s facial reactions. To eliminate the distraction of seeing oneself on video, we simply place a piece of paper over that portion of the screen. Then after a session has concluded, we can turn off recording and find that GoToMeeting has automatically saved the session with the name of the participant for our review later.

GoToMeeting likely works with any remote meeting software, so give it a try with your favorite. It may be able to replace more expensive tools.

iPhone Recording Contraption – Mobile Usability Testing Rig

Check it out, my little hack for recording finger position and screen taps on our prototype.

mobile-testing-rig

This was inspired by a similar design by LevelFive Solutions.

  1. I just took the original, clear plastic case that our iPod Touch came in.
  2. Removed the top of the case.
  3. Plugged in a Hue HD web camera to a USB extension
  4. Duct taped the extension cable to the back. (In a lovely neon orange too!)
  5. Put the iPod Touch in a rubber case.
  6. Put the iPod+Case into the rig.

Now we had a flexible camera boom and a slot that can easily hold our iPod Touch or iPhone for testing. My Galaxy S4 was a bit too big to fit into the case but I’m sure we could find another like case to work with. With a little more time and money we could find a dedicated hard plastic case, like Otterbox, and epoxy the USB extension cable to that.

There are a lot of good, purchasable solutions out there as well, but when you are dealing with no budget and existing parts, sometimes good solutions emerge.

What contraptions have you come up with for testing with mobile devices?

Mega Nav – Bullet Dodged

Starbuck's website has a mega navigational menu.
Starbuck’s website has a mega navigational menu.

While working on a major site overhaul I suggested possibly using a mega nav (designer-daily.com). I thought that it looked pretty and contained a lot more information than traditional navigational menus. Heck, more information and context about your links is better right?

We spent many hours designing our own navigation to look good, provide good context, and pull content in directly from our CMS. It even seemed to test fairly well. Then I attended UIE 17 and listened to Jared Spool (twitter) lambast mega navigations and really drop-down navigation all together.

His argument was one of simplicity. When a visitor to your site views the content they use their eyes (or ears) to understand context and to begin the way-finding process. We all follow what is called the “scent of information”. Much like an animal follows a scent trail to find its pray, we follow information trails to find the content that meets our need.

The main problem with content that reveals itself after a click is that a user doesn’t have a good chance to find that scent trail. If the keywords they are looking for are not visible on the page they are unlikely to find the right link. I wanted to argue that users explore the website with their mouse. But he even had data to show that, in large part, they actually don’t.

Another issue with drop down navigational elements is that most sites don’t provide a clue to the user that there will be animation to display more information. Yes, designers are coming up with more an more ways to hint at animation, and you may test these patterns to discover for yourself which works better. But most people are fully accustomed to the ubiquitous page load. They expect the page load and when the drop-down appears it comes of a bit of a shock. This shock interrupts the scent of information.

After all a page load is intuitive, automatic, and cross-browser compatible! A user expects a link click to cause a page load. When it does occur they can naturally transition to the next page and continue following the scent of information. However, when an animation occurs instead of the expected page load their scent of information is thrown off and they have to start over again. Not the best case.

Also, what are you planning on doing for small mobile devices or touch devices? A mega nav is a bit of a nightmare for mobile users. So, you will have to generate a new page for mobile users that has the same content as the mobile navigation. Doesn’t that seem a bit strange that you force mobile users to a new page that desktop folks don’t see? Wouldn’t it be simpler to just send everyone to that page?

Conclusion: Remove the mega-navigational menu to improve the user experience and provide a more consistent experience across all devices.

After a lot of deliberation around these points we decided to remove the mega-nav and haven’t had an issue yet. Usability testing showed no issues at all in navigation with this change where before we had an 80% success rate.

Bullet dodged! Score one for simplicity. Thank you Jared for opening our eyes, or rather removing the ‘this is cool’ blinders.

What Usability Testing Method Should I Use?

There are several different methods available to test users and usability. There are two dimensions to usability: 1) The location of the participants and 2) the participation level of the moderator. These can be combined into a quadrant to balance the pros and cons of each dimension.

Warning: Use each method as the project requires and time is available. Any user testing is better than no user testing so don’t use this guide as a reason to not do any testing. If there are constraints that restrict use of one tool or another, just test any way you can.

Testing Dimensions:

testing-dimensions

LOCATION OF PARTICIPANT: Local vs. Remote

Local testing is testing where real people are physically present in the same room. The moderator and the users are sitting together. Usually the screen is shown via a projector to keep the watchers from looking over the user’s shoulder.

Pros:

  • Mobile testing is easier, technologically.
  • Body language and reactions can be better recorded.
  • Lower no-show rate.

Cons:

  • Honoraria for participants is high.
  • Bad for short tests.
  • Set up time.
  • Travel expected.

Remote testing is when the moderator and watchers are located physically in a different place from the user. Usually a screen sharing or recording program is used by the participant/user to allow remote people to watch the screen.

Pros:

  • Honoraria not needed or much cheaper.
  • No room requirements.
  • No travel needed.

Cons:

  • Mobile testing is difficult, due to available tools.
  • Higher no-show rate.

PRESENCE OF MODERATOR: Moderated vs. Unmoderated

Moderated testing is where a live person is guiding the participant through the test. Observations are made real time and discussions are possible. Usually these tests have 5-7 participants.

Pros:

  • Usability trends are quickly identified.
  • Interviews allow for flexible testing.
  • Better for testing general usability.
  • Great for longer tests.
  • May have vague end states (Good for multi-situational testing)

Cons:

  • Watchers/Moderators can skew the results easily.
  • One test at a time.
  • Longer to set up (eventually) .

Unmoderated testing is where a computer guides the participant through the test. Observations are made after the fact using data points including: survey results, pass/fail rates, time to completion, and click maps. Usually these tests have 50+ participants.

Pros:

  • Large data sets allow for better trending
  • More accurate data sets
  • Harder to skew the results
  • Simultaneous tests possible
  • Quicker to set up. (eventually)
  • Great for short tests.
  • Able to recruit users in the middle of a task.

Cons:

  • Inflexible testing does not allow for quick changes
  • Rough mock-ups are a waste of time to test with large numbers.
  • Must have well-defined end states. (Participants are in control of moving forward)

Tools:

Local Moderated

  • Paper,
  • Computer,
  • Mobile Device

 Remote Moderated

  • GoToMeeting
  • WebEx
  • etc.

Local Unmoderated

  • none

 Remote Unmoderated

Conclusion

Local moderated testing is great for identifying general trends and overall usability problems. This method should be used near the beginning of a project or anytime a major change needs to be made to an application. Remote unmoderated testing is great for identifying patterns of behavior and for identifying the better of two possibilities. This method should be used to gather trend information for active/live applications and websites. It is also very useful for quickly testing and recruiting for nearly completed prototypes or screenshots.