GoToMeeting, a great tool for remote meetings, is also incredibly useful for moderated usability testing. Using GoToMeeting can help you eliminate the number of observers while still allowing stakeholders to view testing. In addition, the recorded files are small, and the setup is simple.
In recent moderated usability tests, we have started using GoToMeeting to share our sessions. It’s particularly handy for setting up a separate observation room without needing a large lab. We set up a second computer as a guest in a remote viewing room in the GoToMeeting tool, which allows all our observers to come and watch. (We’re careful to mute the observing computer.) This method keeps the number of people watching a session, live and in person, to a minimum. This also keeps the test participant from becoming distracted or intimidated by the large number of observers.
The second and most useful feature is GoToMeeting’s recording option. We can use GoToMeeting to record the screen of the laptop as well as the voices in the room. The file size and quality are both pretty reasonable. For both desktop and mobile testing we can set up a small camera window in the lower corner of the screen that shows the participant’s facial reactions. To eliminate the distraction of seeing oneself on video, we simply place a piece of paper over that portion of the screen. Then after a session has concluded, we can turn off recording and find that GoToMeeting has automatically saved the session with the name of the participant for our review later.
GoToMeeting likely works with any remote meeting software, so give it a try with your favorite. It may be able to replace more expensive tools.
Check it out, my little hack for recording finger position and screen taps on our prototype.
This was inspired by a similar design by LevelFive Solutions.
- I just took the original, clear plastic case that our iPod Touch came in.
- Removed the top of the case.
- Plugged in a Hue HD web camera to a USB extension
- Duct taped the extension cable to the back. (In a lovely neon orange too!)
- Put the iPod Touch in a rubber case.
- Put the iPod+Case into the rig.
Now we had a flexible camera boom and a slot that can easily hold our iPod Touch or iPhone for testing. My Galaxy S4 was a bit too big to fit into the case but I’m sure we could find another like case to work with. With a little more time and money we could find a dedicated hard plastic case, like Otterbox, and epoxy the USB extension cable to that.
There are a lot of good, purchasable solutions out there as well, but when you are dealing with no budget and existing parts, sometimes good solutions emerge.
What contraptions have you come up with for testing with mobile devices?
There are several different methods available to test users and usability. There are two dimensions to usability: 1) The location of the participants and 2) the participation level of the moderator. These can be combined into a quadrant to balance the pros and cons of each dimension.
Warning: Use each method as the project requires and time is available. Any user testing is better than no user testing so don’t use this guide as a reason to not do any testing. If there are constraints that restrict use of one tool or another, just test any way you can.
LOCATION OF PARTICIPANT: Local vs. Remote
Local testing is testing where real people are physically present in the same room. The moderator and the users are sitting together. Usually the screen is shown via a projector to keep the watchers from looking over the user’s shoulder.
- Mobile testing is easier, technologically.
- Body language and reactions can be better recorded.
- Lower no-show rate.
- Honoraria for participants is high.
- Bad for short tests.
- Set up time.
- Travel expected.
Remote testing is when the moderator and watchers are located physically in a different place from the user. Usually a screen sharing or recording program is used by the participant/user to allow remote people to watch the screen.
- Honoraria not needed or much cheaper.
- No room requirements.
- No travel needed.
- Mobile testing is difficult, due to available tools.
- Higher no-show rate.
PRESENCE OF MODERATOR: Moderated vs. Unmoderated
Moderated testing is where a live person is guiding the participant through the test. Observations are made real time and discussions are possible. Usually these tests have 5-7 participants.
- Usability trends are quickly identified.
- Interviews allow for flexible testing.
- Better for testing general usability.
- Great for longer tests.
- May have vague end states (Good for multi-situational testing)
- Watchers/Moderators can skew the results easily.
- One test at a time.
- Longer to set up (eventually) .
Unmoderated testing is where a computer guides the participant through the test. Observations are made after the fact using data points including: survey results, pass/fail rates, time to completion, and click maps. Usually these tests have 50+ participants.
- Large data sets allow for better trending
- More accurate data sets
- Harder to skew the results
- Simultaneous tests possible
- Quicker to set up. (eventually)
- Great for short tests.
- Able to recruit users in the middle of a task.
- Inflexible testing does not allow for quick changes
- Rough mock-ups are a waste of time to test with large numbers.
- Must have well-defined end states. (Participants are in control of moving forward)
- Mobile Device
Local moderated testing is great for identifying general trends and overall usability problems. This method should be used near the beginning of a project or anytime a major change needs to be made to an application. Remote unmoderated testing is great for identifying patterns of behavior and for identifying the better of two possibilities. This method should be used to gather trend information for active/live applications and websites. It is also very useful for quickly testing and recruiting for nearly completed prototypes or screenshots.