[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.03,0:00:05.08,Default,,0000,0000,0000,,In this video we are going to introduce a technique called Heuristic Evaluation.
Dialogue: 0,0:00:05.08,0:00:11.05,Default,,0000,0000,0000,,As we talked about at the beginning of the course, there’s lots of different ways to evaluate software.
Dialogue: 0,0:00:11.05,0:00:14.06,Default,,0000,0000,0000,,One that you might be most familiar with is empirical methods,
Dialogue: 0,0:00:14.06,0:00:19.04,Default,,0000,0000,0000,,where, of some level of formality, you have actual people trying out your software.
Dialogue: 0,0:00:19.04,0:00:25.03,Default,,0000,0000,0000,,It’s also possible to have formal methods, where you’re building a model
Dialogue: 0,0:00:25.03,0:00:28.02,Default,,0000,0000,0000,,of how people behave in a particular situation,
Dialogue: 0,0:00:28.02,0:00:32.02,Default,,0000,0000,0000,,and that enables you to predict how different user interfaces will work.
Dialogue: 0,0:00:32.02,0:00:36.03,Default,,0000,0000,0000,,Or, if you can’t build a closed-form formal model,
Dialogue: 0,0:00:36.03,0:00:40.05,Default,,0000,0000,0000,,you can also try out your interface with simulation and have automated tests —
Dialogue: 0,0:00:40.05,0:00:44.08,Default,,0000,0000,0000,,that can detect usability bugs and effective designs.
Dialogue: 0,0:00:44.08,0:00:49.08,Default,,0000,0000,0000,,This works especially well for low-level stuff; it’s harder to do for higher-level stuff.
Dialogue: 0,0:00:49.08,0:00:52.09,Default,,0000,0000,0000,,And what we’re going to talk about today is critique-based approaches,
Dialogue: 0,0:00:52.09,0:01:00.00,Default,,0000,0000,0000,,where people are giving you feedback directly, based on their expertise or a set of heuristics.
Dialogue: 0,0:01:00.00,0:01:03.02,Default,,0000,0000,0000,,As any of you who have ever taken an art or design class know,
Dialogue: 0,0:01:03.02,0:01:06.07,Default,,0000,0000,0000,,peer critique can be an incredibly effective form of feedback,
Dialogue: 0,0:01:06.07,0:01:09.02,Default,,0000,0000,0000,,and it can make you make your designs even better.
Dialogue: 0,0:01:09.02,0:01:12.10,Default,,0000,0000,0000,,You can get peer critique really at any stage of your design process,
Dialogue: 0,0:01:12.10,0:01:16.84,Default,,0000,0000,0000,,but I’d like to highlight a couple that I think can be particularly valuable.
Dialogue: 0,0:01:16.84,0:01:21.39,Default,,0000,0000,0000,,First, it’s really valuable to get peer critique before user testing,
Dialogue: 0,0:01:21.39,0:01:27.08,Default,,0000,0000,0000,,because that helps you not waste your users on stuff that’s just going to get picked up automatically.
Dialogue: 0,0:01:27.08,0:01:30.35,Default,,0000,0000,0000,,You want to be able to focus the valuable resources of user testing
Dialogue: 0,0:01:30.35,0:01:34.19,Default,,0000,0000,0000,,on stuff that other people wouldn’t be able to pick up on.
Dialogue: 0,0:01:34.19,0:01:37.02,Default,,0000,0000,0000,,The rich qualitative feedback that peer critique provides
Dialogue: 0,0:01:37.02,0:01:40.78,Default,,0000,0000,0000,,can also be really valuable before redesigning your application,
Dialogue: 0,0:01:40.78,0:01:45.15,Default,,0000,0000,0000,,because what it can do is it can show you what parts of your app you probably want to keep,
Dialogue: 0,0:01:45.15,0:01:48.78,Default,,0000,0000,0000,,and what are other parts that are more problematic and deserve redesign.
Dialogue: 0,0:01:49.43,0:01:51.14,Default,,0000,0000,0000,,Third, sometimes, you know there are problems,
Dialogue: 0,0:01:51.14,0:01:55.66,Default,,0000,0000,0000,,and you need data to be able to convince other stakeholders to make the changes.
Dialogue: 0,0:01:55.66,0:01:59.55,Default,,0000,0000,0000,,And peer critique can be a great way, especially if it’s structured,
Dialogue: 0,0:01:59.55,0:02:04.63,Default,,0000,0000,0000,,to be able to get the feedback that you need, to make the changes that you know need to happen.
Dialogue: 0,0:02:05.57,0:02:11.01,Default,,0000,0000,0000,,And lastly, this kind of structured peer critique can be really valuable before releasing software,
Dialogue: 0,0:02:11.01,0:02:15.63,Default,,0000,0000,0000,,because it helps you do a final sanding of the entire design, and smooth out any rough edges.
Dialogue: 0,0:02:15.63,0:02:20.79,Default,,0000,0000,0000,,As with most types of evaluation, it’s usually helpful to begin with a clear goal,
Dialogue: 0,0:02:20.79,0:02:24.07,Default,,0000,0000,0000,,even if what you ultimately learn is completely unexpected.
Dialogue: 0,0:02:26.00,0:02:30.66,Default,,0000,0000,0000,,And so, what we’re going to talk about today is a particular technique called Heuristic Evaluation.
Dialogue: 0,0:02:30.66,0:02:35.46,Default,,0000,0000,0000,,Heuristic Evaluation was created by Jakob Nielsen and colleagues, about twenty years ago now.
Dialogue: 0,0:02:36.12,0:02:41.65,Default,,0000,0000,0000,,And the goal of Heuristic Evaluation is to be able to find usability problems in the design.
Dialogue: 0,0:02:42.65,0:02:44.45,Default,,0000,0000,0000,,I first learned about Heuristic Evaluation
Dialogue: 0,0:02:44.45,0:02:49.75,Default,,0000,0000,0000,,when I TA’d James Landay’s Intro to HCI course, and I’ve been using it and teaching it ever since.
Dialogue: 0,0:02:49.75,0:02:54.05,Default,,0000,0000,0000,,It’s a really valuable technique because it lets you get feedback really quickly
Dialogue: 0,0:02:54.05,0:02:57.70,Default,,0000,0000,0000,,and it’s a high bang-for-the-buck strategy.
Dialogue: 0,0:02:57.70,0:03:01.59,Default,,0000,0000,0000,,And the slides that I have here are based off James’ slides for this course,
Dialogue: 0,0:03:01.59,0:03:05.90,Default,,0000,0000,0000,,and the materials are all available on Jacob Nielsen’s website.
Dialogue: 0,0:03:05.90,0:03:10.06,Default,,0000,0000,0000,,The basic idea of heuristic evaluation is that you’re going to provide a set of people —
Dialogue: 0,0:03:10.06,0:03:15.04,Default,,0000,0000,0000,,often other stakeholders on the design team or outside design experts —
Dialogue: 0,0:03:15.04,0:03:17.94,Default,,0000,0000,0000,,with a set of heuristics or principles,
Dialogue: 0,0:03:17.94,0:03:22.83,Default,,0000,0000,0000,,and they’re going to use those to look for problems in your design.
Dialogue: 0,0:03:23.57,0:03:26.10,Default,,0000,0000,0000,,Each of them is first going to do this independently
Dialogue: 0,0:03:26.10,0:03:31.06,Default,,0000,0000,0000,,and so they’ll walk through a variety of tasks using your design to look for these bugs.
Dialogue: 0,0:03:32.56,0:03:36.85,Default,,0000,0000,0000,,And you’ll see that different evaluators are going to find different problems.
Dialogue: 0,0:03:36.85,0:03:41.33,Default,,0000,0000,0000,,And then they’re going to communicate and talk together only at the end, afterwards.
Dialogue: 0,0:03:43.07,0:03:47.27,Default,,0000,0000,0000,,At the end of the process, they’re going to get back together and talk about what they found.
Dialogue: 0,0:03:47.27,0:03:50.59,Default,,0000,0000,0000,,And this “independent first, gather afterwards”
Dialogue: 0,0:03:50.59,0:03:56.58,Default,,0000,0000,0000,,is how you get a “wisdom of crowds” benefit in having multiple evaluators.
Dialogue: 0,0:03:56.58,0:03:58.77,Default,,0000,0000,0000,,And one of the reasons that we’re talking about this early in the class
Dialogue: 0,0:03:58.77,0:04:05.07,Default,,0000,0000,0000,,is that it’s a technique that you can use, either on a working user interface or on sketches of user interfaces.
Dialogue: 0,0:04:05.07,0:04:10.42,Default,,0000,0000,0000,,And so heuristic evaluation works really well in conjunction with paper prototypes
Dialogue: 0,0:04:10.42,0:04:16.35,Default,,0000,0000,0000,,and other rapid, low fidelity techniques that you may be using to get your design ideas out quick and fast.
Dialogue: 0,0:04:18.29,0:04:22.38,Default,,0000,0000,0000,,Here’s Neilsen’s ten heuristics, and they’re a pretty darn good set.
Dialogue: 0,0:04:22.38,0:04:25.04,Default,,0000,0000,0000,,That said, there’s nothing magic about these heuristics.
Dialogue: 0,0:04:25.04,0:04:30.30,Default,,0000,0000,0000,,They do a pretty good job of covering many of the problems that you’ll see in many user interfaces;
Dialogue: 0,0:04:30.30,0:04:33.49,Default,,0000,0000,0000,,but you can add on any that you want
Dialogue: 0,0:04:33.49,0:04:37.61,Default,,0000,0000,0000,,and get rid of any that aren’t appropriate for your system.
Dialogue: 0,0:04:37.61,0:04:40.98,Default,,0000,0000,0000,,We’re going to go over the content of these ten heuristics in the next couple lectures,
Dialogue: 0,0:04:40.98,0:04:45.54,Default,,0000,0000,0000,,and in this lecture I’d like to introduce the process that you’re going to use with these heuristics.
Dialogue: 0,0:04:46.34,0:04:49.24,Default,,0000,0000,0000,,So here’s what you’re going to have your evaluators do:
Dialogue: 0,0:04:49.24,0:04:52.27,Default,,0000,0000,0000,,Give them a couple of tasks to use your design for,
Dialogue: 0,0:04:52.27,0:04:57.02,Default,,0000,0000,0000,,and have them do each task, stepping through carefully several times.
Dialogue: 0,0:04:57.02,0:05:00.58,Default,,0000,0000,0000,,When they’re doing this, they’re going to keep the list of usability principles
Dialogue: 0,0:05:00.58,0:05:03.06,Default,,0000,0000,0000,,as a reminder of things to pay attention to.
Dialogue: 0,0:05:03.06,0:05:05.71,Default,,0000,0000,0000,,Now which principles will you use?
Dialogue: 0,0:05:05.71,0:05:08.75,Default,,0000,0000,0000,,I think Nielsen’s ten heuristics are a fantastic start,
Dialogue: 0,0:05:08.75,0:05:12.96,Default,,0000,0000,0000,,and you can augment those with anything else that’s relevant for your domain.
Dialogue: 0,0:05:12.96,0:05:19.04,Default,,0000,0000,0000,,So, if you have particular design goals that you would like your design to achieve, include those in the list.
Dialogue: 0,0:05:19.04,0:05:21.57,Default,,0000,0000,0000,,Or, if you have particular goals that you’ve set up
Dialogue: 0,0:05:21.57,0:05:25.89,Default,,0000,0000,0000,,from competitive analysis of designs that are out there already,
Dialogue: 0,0:05:25.89,0:05:27.31,Default,,0000,0000,0000,,that’s great too.
Dialogue: 0,0:05:27.31,0:05:32.62,Default,,0000,0000,0000,,Or if there are things that you’ve seen your or other designs excel at,
Dialogue: 0,0:05:32.62,0:05:37.19,Default,,0000,0000,0000,,those are important goals too and can be included in your list of heuristics.
Dialogue: 0,0:05:38.84,0:05:42.71,Default,,0000,0000,0000,,And then obviously, the important part is that you’re going to take what you learn from these evaluators
Dialogue: 0,0:05:42.71,0:05:48.61,Default,,0000,0000,0000,,and use those violations of the heuristics as a way of fixing problems and redesigning.
Dialogue: 0,0:05:49.36,0:05:55.04,Default,,0000,0000,0000,,Let’s talk a little bit more about why you might want to have multiple evaluators rather than just one.
Dialogue: 0,0:05:55.04,0:05:59.90,Default,,0000,0000,0000,,The graph on this slide is adapted from Jacob Neilsen’s work on heuristic evaluation
Dialogue: 0,0:05:59.90,0:06:06.57,Default,,0000,0000,0000,,and what you see is each black square is a bug that a particular evaluator found.
Dialogue: 0,0:06:07.78,0:06:11.91,Default,,0000,0000,0000,,An individual evaluator represents a row of this matrix
Dialogue: 0,0:06:11.91,0:06:15.04,Default,,0000,0000,0000,,and there’s about twenty evaluators in this set.
Dialogue: 0,0:06:15.04,0:06:16.97,Default,,0000,0000,0000,,The columns represent the problems.
Dialogue: 0,0:06:16.97,0:06:21.57,Default,,0000,0000,0000,,And what you can see is that there’s some problems that were found by relatively few evaluators
Dialogue: 0,0:06:21.57,0:06:24.62,Default,,0000,0000,0000,,and other stuff which almost everybody found.
Dialogue: 0,0:06:24.62,0:06:29.06,Default,,0000,0000,0000,,So we’re going to call the stuff on the right the easy problems and the stuff on the left hard problems.
Dialogue: 0,0:06:30.09,0:06:35.00,Default,,0000,0000,0000,,And so, in aggregate, what we can say is that no evaluator found every problem,
Dialogue: 0,0:06:35.00,0:06:41.41,Default,,0000,0000,0000,,and some evaluators found more than others, and so there are better and worse people to do this.
Dialogue: 0,0:06:43.01,0:06:44.95,Default,,0000,0000,0000,,So why not have lots of evaluators?
Dialogue: 0,0:06:44.95,0:06:48.88,Default,,0000,0000,0000,,Well, as you add more evaluators, they do find more problems;
Dialogue: 0,0:06:49.62,0:06:53.16,Default,,0000,0000,0000,,but it kind of tapers off over time — you lose that benefit eventually.
Dialogue: 0,0:06:53.54,0:06:58.44,Default,,0000,0000,0000,,And so from a cost-benefit perspective it’s just stops making sense after a certain point.
Dialogue: 0,0:06:59.04,0:07:00.60,Default,,0000,0000,0000,,So where’s the peak of this curve?
Dialogue: 0,0:07:00.60,0:07:04.13,Default,,0000,0000,0000,,It’s of course going to depend on the user interface that you’re working with,
Dialogue: 0,0:07:04.13,0:07:08.47,Default,,0000,0000,0000,,how much you’re paying people, how much time is involved — all sorts of factors.
Dialogue: 0,0:07:08.47,0:07:13.46,Default,,0000,0000,0000,,Jakob Nielsen’s rule of thumb for these kinds of user interfaces and heuristic evaluation
Dialogue: 0,0:07:13.46,0:07:19.03,Default,,0000,0000,0000,,is that three to five people tends to work pretty well; and that’s been my experience too.
Dialogue: 0,0:07:20.17,0:07:24.02,Default,,0000,0000,0000,,And I think that definitely one of the reasons that people use heuristic evaluation
Dialogue: 0,0:07:24.02,0:07:28.04,Default,,0000,0000,0000,,is because it can be an extremely cost-effective way of finding problems.
Dialogue: 0,0:07:29.12,0:07:31.59,Default,,0000,0000,0000,,In one study that Jacob Nielsen ran,
Dialogue: 0,0:07:31.59,0:07:37.29,Default,,0000,0000,0000,,he estimated that the cost of the problems found with heuristic evaluation were $500,000
Dialogue: 0,0:07:37.29,0:07:41.17,Default,,0000,0000,0000,,and the cost of performing it was just over $10,000,
Dialogue: 0,0:07:41.17,0:07:48.98,Default,,0000,0000,0000,,and so he estimates a 48-fold benefit-cost ratio for this particular user interface.
Dialogue: 0,0:07:48.98,0:07:54.91,Default,,0000,0000,0000,,Obviously, these numbers are back of the envelope, and your mileage will vary.
Dialogue: 0,0:07:54.91,0:07:58.98,Default,,0000,0000,0000,,You can think about how to estimate the benefit that you get from something like this
Dialogue: 0,0:07:58.98,0:08:03.30,Default,,0000,0000,0000,,if you have an in-house software tool using something like productivity increases —
Dialogue: 0,0:08:03.30,0:08:06.96,Default,,0000,0000,0000,,that, if you are making an expense reporting system
Dialogue: 0,0:08:06.96,0:08:11.67,Default,,0000,0000,0000,,or other in-house system that will make people’s time more efficiently used —
Dialogue: 0,0:08:11.67,0:08:13.90,Default,,0000,0000,0000,,that’s a big usability win.
Dialogue: 0,0:08:13.90,0:08:17.54,Default,,0000,0000,0000,,And if you’ve got software that you’re making available on the open market,
Dialogue: 0,0:08:17.54,0:08:22.45,Default,,0000,0000,0000,,you can think about the benefit from sales or other measures like that.
Dialogue: 0,0:08:23.60,0:08:28.26,Default,,0000,0000,0000,,One thing that we can get from that graph is that evaluators are more likely to find severe problems
Dialogue: 0,0:08:28.26,0:08:29.62,Default,,0000,0000,0000,,and that’s good news;
Dialogue: 0,0:08:29.62,0:08:32.26,Default,,0000,0000,0000,,and so with a relatively small number of people,
Dialogue: 0,0:08:32.26,0:08:35.91,Default,,0000,0000,0000,,you’re pretty likely to stumble across the most important stuff.
Dialogue: 0,0:08:35.91,0:08:40.93,Default,,0000,0000,0000,,However, as we saw with just one person in this particular case,
Dialogue: 0,0:08:40.93,0:08:46.11,Default,,0000,0000,0000,,even the best evaluator found only about a third of the problems of the system.
Dialogue: 0,0:08:46.11,0:08:50.68,Default,,0000,0000,0000,,And so that’s why ganging up a number of evaluators, say five,
Dialogue: 0,0:08:50.68,0:08:54.97,Default,,0000,0000,0000,,is going to get you most of the benefit that you’ll be going to be able to achieve.
Dialogue: 0,0:08:55.96,0:09:00.02,Default,,0000,0000,0000,,If we compare heuristic evaluation and user testing, one of the things that we see
Dialogue: 0,0:09:00.02,0:09:06.93,Default,,0000,0000,0000,,is that heuristic evaluation can often be a lot faster — It takes just an hour or two for an evaluator —
Dialogue: 0,0:09:06.93,0:09:11.46,Default,,0000,0000,0000,,and the mechanics of getting a user test up and running can take longer,
Dialogue: 0,0:09:11.46,0:09:16.34,Default,,0000,0000,0000,,not even accounting for the fact that you may have to build software.
Dialogue: 0,0:09:17.67,0:09:21.46,Default,,0000,0000,0000,,Also, the heuristic evaluation results come pre-interpreted
Dialogue: 0,0:09:21.46,0:09:26.16,Default,,0000,0000,0000,,because your evaluators are directly providing you with problems and things to fix,
Dialogue: 0,0:09:26.16,0:09:34.32,Default,,0000,0000,0000,,and so it saves you the time of having to infer from the usability tests what might be the problem or solution.
Dialogue: 0,0:09:35.64,0:09:39.24,Default,,0000,0000,0000,,Now conversely, experts walking through your system
Dialogue: 0,0:09:39.24,0:09:44.10,Default,,0000,0000,0000,,can generate false positives that wouldn’t actually happen in a real environment.
Dialogue: 0,0:09:44.10,0:09:50.38,Default,,0000,0000,0000,,And this indeed does happen, and so user testing is, sort of, by definition going to be more accurate.
Dialogue: 0,0:09:52.10,0:09:55.07,Default,,0000,0000,0000,,At the end of the day I think it’s valuable to alternate methods:
Dialogue: 0,0:09:55.07,0:10:00.31,Default,,0000,0000,0000,,All of the different techniques that you’ll learn in this class for getting feedback can each be valuable,
Dialogue: 0,0:10:00.31,0:10:04.85,Default,,0000,0000,0000,,and that [by] cycling through them you can often get the benefits of each.
Dialogue: 0,0:10:04.85,0:10:10.64,Default,,0000,0000,0000,,And that can be because with user evaluation and user testing, you’ll find different problems,
Dialogue: 0,0:10:10.64,0:10:15.49,Default,,0000,0000,0000,,and by running HE or something like that early in the design process,
Dialogue: 0,0:10:15.49,0:10:20.22,Default,,0000,0000,0000,,you’ll avoid wasting real users that you may bring in later on.
Dialogue: 0,0:10:21.48,0:10:24.94,Default,,0000,0000,0000,,So now that we’ve seen the benefits, what are the steps?
Dialogue: 0,0:10:24.94,0:10:29.64,Default,,0000,0000,0000,,The first thing to do is to get all of your evaluators up to speed,
Dialogue: 0,0:10:29.64,0:10:35.80,Default,,0000,0000,0000,,on what the story is behind your software — any necessary domain knowledge they might need —
Dialogue: 0,0:10:35.81,0:10:39.66,Default,,0000,0000,0000,,and tell them about the scenario that you’re going to have them step through.
Dialogue: 0,0:10:40.88,0:10:45.08,Default,,0000,0000,0000,,Then obviously, you have the evaluation phase where people are working through the interface.
Dialogue: 0,0:10:45.08,0:10:50.08,Default,,0000,0000,0000,,Afterwards, each person is going to assign a severity rating,
Dialogue: 0,0:10:50.08,0:10:52.74,Default,,0000,0000,0000,,and you do this individually first,
Dialogue: 0,0:10:52.74,0:10:56.12,Default,,0000,0000,0000,,and then you’re going to aggregate those into a group severity rating
Dialogue: 0,0:10:56.12,0:10:59.50,Default,,0000,0000,0000,,and produce an aggregate report out of that.
Dialogue: 0,0:11:00.69,0:11:06.28,Default,,0000,0000,0000,,And finally, once you’ve got this aggregated report, you can share that with the design team,
Dialogue: 0,0:11:06.28,0:11:09.72,Default,,0000,0000,0000,,and the design team can discuss what to do with that.
Dialogue: 0,0:11:10.01,0:11:12.91,Default,,0000,0000,0000,,Doing this kind of expert review can be really taxing,
Dialogue: 0,0:11:12.91,0:11:16.10,Default,,0000,0000,0000,,and so for each of the scenarios that you lay out in your design,
Dialogue: 0,0:11:16.10,0:11:22.06,Default,,0000,0000,0000,,it can be valuable to have the evaluator go through that scenario twice.
Dialogue: 0,0:11:22.06,0:11:28.03,Default,,0000,0000,0000,,The first time, they’ll just get a sense of it; and the second time, they can focus on more specific elements.
Dialogue: 0,0:11:30.03,0:11:34.71,Default,,0000,0000,0000,,If you’ve got some walk-up-and-use system, like a ticket machine somewhere,
Dialogue: 0,0:11:34.71,0:11:38.90,Default,,0000,0000,0000,,then you may want to not give people any background information at all,
Dialogue: 0,0:11:38.90,0:11:42.10,Default,,0000,0000,0000,,because if you’ve got people that are just getting off the bus or the train,
Dialogue: 0,0:11:42.10,0:11:45.37,Default,,0000,0000,0000,,and they walk up to your machine without any prior information,
Dialogue: 0,0:11:45.37,0:11:49.35,Default,,0000,0000,0000,,that’s the experience you’ll want your evaluators to have.
Dialogue: 0,0:11:49.35,0:11:53.48,Default,,0000,0000,0000,,On the other hand, if you’re going to have a genomic system or other expert user interface,
Dialogue: 0,0:11:53.48,0:11:57.02,Default,,0000,0000,0000,,you’ll want to to make sure that whatever training you would give to real users,
Dialogue: 0,0:11:57.02,0:11:59.57,Default,,0000,0000,0000,,you’re going to give to your evaluators as well.
Dialogue: 0,0:11:59.57,0:12:03.55,Default,,0000,0000,0000,,In other words, whatever the background is, it should be realistic.
Dialogue: 0,0:12:05.74,0:12:08.65,Default,,0000,0000,0000,,When your evaluators are walking through your interface,
Dialogue: 0,0:12:08.65,0:12:12.57,Default,,0000,0000,0000,,it’s going to be important to produce a list of very specific problems
Dialogue: 0,0:12:12.57,0:12:16.98,Default,,0000,0000,0000,,and explain those problems with regard to one of the design heuristics.
Dialogue: 0,0:12:16.98,0:12:21.20,Default,,0000,0000,0000,,You don’t want people to just to be, like, “I don’t like it.”
Dialogue: 0,0:12:21.20,0:12:26.23,Default,,0000,0000,0000,,And in order to maxilinearly preach you these results for the design team;
Dialogue: 0,0:12:26.23,0:12:31.44,Default,,0000,0000,0000,,you’ll want to list each one of these separately so that they can be dealt with efficiently.
Dialogue: 0,0:12:31.44,0:12:37.16,Default,,0000,0000,0000,,Separate listings can also help you avoid listing the same repeated problem over and over again.
Dialogue: 0,0:12:37.16,0:12:42.48,Default,,0000,0000,0000,,If there’s a repeated element on every single screen, you don’t want to list it at every single screen;
Dialogue: 0,0:12:42.48,0:12:45.82,Default,,0000,0000,0000,,you want to list it once so that it can be fixed once.
Dialogue: 0,0:12:46.88,0:12:52.32,Default,,0000,0000,0000,,And these problems can be very detailed, like “the name of something is confusing,”
Dialogue: 0,0:12:52.32,0:12:55.71,Default,,0000,0000,0000,,or it can be something that has to do more with the flow of the user interface,
Dialogue: 0,0:12:55.71,0:13:02.11,Default,,0000,0000,0000,,or the architecture of the user experience and that’s not specifically tied to an interface element.
Dialogue: 0,0:13:03.23,0:13:07.05,Default,,0000,0000,0000,,Your evaluators may also find that something is missing that ought to be there,
Dialogue: 0,0:13:07.05,0:13:11.25,Default,,0000,0000,0000,,and this can be sometime ambiguous with early prototypes, like paper prototypes.
Dialogue: 0,0:13:11.25,0:13:17.36,Default,,0000,0000,0000,,And so you’ll want to clarify whether the user interface is something that you believe to be complete,
Dialogue: 0,0:13:17.36,0:13:21.76,Default,,0000,0000,0000,,or whether there are intentional elements missing ahead of time.
Dialogue: 0,0:13:22.18,0:13:25.79,Default,,0000,0000,0000,,And, of course, sometimes there are features that are going to be obviously there
Dialogue: 0,0:13:25.79,0:13:28.08,Default,,0000,0000,0000,,that are implied by the user interface.
Dialogue: 0,0:13:28.08,0:13:31.89,Default,,0000,0000,0000,,And so, mellow out, and relax on those.
Dialogue: 0,0:13:34.51,0:13:36.76,Default,,0000,0000,0000,,After your evaluators have gone through the interface,
Dialogue: 0,0:13:36.76,0:13:41.26,Default,,0000,0000,0000,,they can each independently assign a severity rating to all of the problems that they’ve found.
Dialogue: 0,0:13:41.26,0:13:45.10,Default,,0000,0000,0000,,And that’s going to enable you to allocate resources to fix those problems.
Dialogue: 0,0:13:45.10,0:13:48.22,Default,,0000,0000,0000,,It can also help give you feedback about how well you’re doing
Dialogue: 0,0:13:48.22,0:13:50.97,Default,,0000,0000,0000,,in terms of the usability of your system in general,
Dialogue: 0,0:13:50.97,0:13:55.18,Default,,0000,0000,0000,,and give you a kind of a benchmark of your efforts in this vein.
Dialogue: 0,0:13:56.38,0:14:01.12,Default,,0000,0000,0000,,The severity measure that your evaluators are going to come up with is going to combine several things:
Dialogue: 0,0:14:01.12,0:14:05.03,Default,,0000,0000,0000,,It’s going to combine the frequency, the impact,
Dialogue: 0,0:14:05.03,0:14:08.93,Default,,0000,0000,0000,,and the pervasiveness of the problem that they’re seeing on the screen.
Dialogue: 0,0:14:08.93,0:14:14.05,Default,,0000,0000,0000,,So, something that is only in one place may be a less big deal
Dialogue: 0,0:14:14.05,0:14:18.56,Default,,0000,0000,0000,,than something that shows up throughout the entire user interface.
Dialogue: 0,0:14:18.56,0:14:23.02,Default,,0000,0000,0000,,Similarly, there are going to be some things like misaligned text,
Dialogue: 0,0:14:23.02,0:14:27.55,Default,,0000,0000,0000,,which may be inelegant, but aren’t a deal killer in terms of your software.
Dialogue: 0,0:14:29.06,0:14:34.44,Default,,0000,0000,0000,,And here is the severity rating system that Nielsen created; you can obviously use anything that you want:
Dialogue: 0,0:14:34.44,0:14:36.69,Default,,0000,0000,0000,,It ranges from zero to four,
Dialogue: 0,0:14:36.69,0:14:41.90,Default,,0000,0000,0000,,where zero is “at the end of the day your evaluator decides it actually is not usability problem,”
Dialogue: 0,0:14:41.90,0:14:47.72,Default,,0000,0000,0000,,all the way up to it being something really catastrophic that has to get fixed right away.
Dialogue: 0,0:14:48.77,0:14:51.34,Default,,0000,0000,0000,,And here is an example of a particular problem
Dialogue: 0,0:14:51.34,0:14:56.03,Default,,0000,0000,0000,,that our TA Robby found when he was taking CS147 as a student.
Dialogue: 0,0:14:56.03,0:15:01.08,Default,,0000,0000,0000,,He walked through somebody’s mobile interface that had a “weight” entry element to it;
Dialogue: 0,0:15:01.08,0:15:05.92,Default,,0000,0000,0000,,and he realized that once you entered your weight, there is no way to edit it after the fact.
Dialogue: 0,0:15:05.92,0:15:12.26,Default,,0000,0000,0000,,So, that’s kind of clunky, you wish you could fix it — maybe not a disaster.
Dialogue: 0,0:15:12.26,0:15:17.08,Default,,0000,0000,0000,,And so what you see here is he’s listed the issue, he’s given it a severity rating,
Dialogue: 0,0:15:17.08,0:15:23.16,Default,,0000,0000,0000,,he’s got the heuristic that it violates, and then he describes exactly what the problem is.
Dialogue: 0,0:15:23.63,0:15:26.87,Default,,0000,0000,0000,,And finally, after all your evaluators have gone through the interface,
Dialogue: 0,0:15:26.87,0:15:31.27,Default,,0000,0000,0000,,listed their problems, and combined them in terms of the severity and importance,
Dialogue: 0,0:15:31.27,0:15:34.18,Default,,0000,0000,0000,,you’ll want to debrief with the design team.
Dialogue: 0,0:15:34.18,0:15:39.17,Default,,0000,0000,0000,,This is a nice chance to be able to discuss general issues in the user interface and qualitative feedback,
Dialogue: 0,0:15:39.17,0:15:42.23,Default,,0000,0000,0000,,and it gives you a chance to go through each of these items
Dialogue: 0,0:15:42.23,0:15:45.68,Default,,0000,0000,0000,,and suggest improvements on how you can address these problems.
Dialogue: 0,0:15:47.71,0:15:51.10,Default,,0000,0000,0000,,In this debrief session, it can be valuable for the development team
Dialogue: 0,0:15:51.10,0:15:55.91,Default,,0000,0000,0000,,to estimate the amount of effort that it would take to fix one of these problems.
Dialogue: 0,0:15:55.91,0:16:01.44,Default,,0000,0000,0000,,So, for example, if you’ve got something that is one on your severity scale and not too big a deal —
Dialogue: 0,0:16:01.44,0:16:06.13,Default,,0000,0000,0000,,it might have something to do with wording and its dirt simple to fix —
Dialogue: 0,0:16:06.13,0:16:08.34,Default,,0000,0000,0000,,that tells you “go ahead and fix it.”
Dialogue: 0,0:16:08.34,0:16:11.15,Default,,0000,0000,0000,,Conversely, you may having something which is a catastrophe
Dialogue: 0,0:16:11.15,0:16:15.48,Default,,0000,0000,0000,,which takes a lot more effort, but its importance will lead you to fix it.
Dialogue: 0,0:16:15.48,0:16:19.60,Default,,0000,0000,0000,,And there’s other things where the importance relative to the cost involved
Dialogue: 0,0:16:19.60,0:16:22.81,Default,,0000,0000,0000,,just don’t make sense to deal with right now.
Dialogue: 0,0:16:22.81,0:16:26.87,Default,,0000,0000,0000,,And this debrief session can be a great way to brainstorm future design ideas,
Dialogue: 0,0:16:26.87,0:16:29.72,Default,,0000,0000,0000,,especially while you’ve got all the stakeholders in the room,
Dialogue: 0,0:16:29.72,0:16:34.37,Default,,0000,0000,0000,,and the ideas about what the issues are with the user interface are fresh in their minds.
Dialogue: 0,0:16:34.37,0:16:40.75,Default,,0000,0000,0000,,In the next two videos we’ll go through Neilsons’ ten heuristics and talk more about what they mean.