Home → Articles → Usability Testing Pitfalls (page 3)

Usability Testing: Lessons Learned

Overall, our usability tests were in fact helpful in finding some problems, and also helpful in giving everybody (both dev team and users) a sense for whether the software was generally what people would want. Those are good things to investigate before releasing, and to our credit, we did those. (All too often, people use the release date as the date to find out whether the users will like the app.) But it's clear that our usability tests fell short in some important respects. Here are the lessons I drew from the experience:

• Even if you seem to have a pretty strong sense for the features you need to implement, if you don't understand the user tasks they support then you are less likely to correctly support those tasks. In our case, we mistook "Pull up the Lead Cost report for March 2005" for a user task. The real user task was "Balance the budget for March 2005". The users had no problem pulling up the Lead Cost report. But they would have had a serious problem balancing the budget had we asked them to do that.
• To get at the real tasks, you need to perform some kind of up-front discovery process, such as contextual inquiry. Software requirements solicitation, without a discussion of how required features will be used, misses the tasks. While I suppose it's theoretically possible that one could produce a complete set of requirements (i.e., one that makes understanding the context of use irrelevant), I can with some confidence say that in the history of the world, this has never actually happened except for the most trivial of software applications. It is useful for the development team to understand the tasks before trying to support them with software. There is a lot of value in instilling the development team with an appropriate set of intuitions about the supported work.
• Even if the users have no problems during the usability tests, that doesn't mean that there aren't any. If you miss important tasks, the users can't find problems with them.

Post a comment

Your name:

Your e-mail address (won't be displayed):

Your web site (optional):

example: www.xyz.com

Your comment:

Preview:

By You

Please help us reduce comment spam: