Return to Video

Bug Triage - Software Testing

  • 0:00 - 0:04
    A bug triage is the process by which the severity of different bugs is determined,
  • 0:04 - 0:06
    and we start to disambiguate between different bugs
  • 0:06 - 0:09
    in order to basically try to get a handle on which bugs we can report in parallel.
  • 0:09 - 0:13
    The deal is that any inputs that trigger separate bugs can be reported in parallel,
  • 0:13 - 0:16
    but if we report all of the bug-triggering inputs that we found,
  • 0:16 - 0:18
    what we're going to do is be causing a lot of duplicate bug reports.
  • 0:18 - 0:23
    One of the golden rules of reporting bugs--and I'll go over this a little bit later in more detail--
  • 0:23 - 0:25
    is that you don't report duplicate bugs.
  • 0:25 - 0:27
    This wears on developers incredibly much,
  • 0:27 - 0:29
    and they'll soon get tired of you and start ignoring you.
  • 0:29 - 0:32
    I say this from experience. I've reported quite a few duplicate bugs myself.
  • 0:32 - 0:34
    It's not that fun. People don't like it.
  • 0:34 - 0:36
    The question now is what does bug triage mean.
  • 0:36 - 0:38
    How do we start getting a handle on which bug-triggering inputs
  • 0:38 - 0:41
    map to different bugs and which ones map to the same bugs.
  • 0:41 - 0:43
    Basically, there is no silver bullet here.
  • 0:43 - 0:46
    What we have is we're going to see there is a number of different weapons
  • 0:46 - 0:48
    we can use to disambiguate bugs.
  • 0:48 - 0:53
    In an easy-as-possible case the bugs in the system are causing assertion violation messages.
  • 0:53 - 0:57
    One thing we can do is disambiguate based on assertion violations.
  • 0:57 - 0:59
    What I basically mean here is that you do what I gave a demonstration of doing
  • 0:59 - 1:03
    a little bit earlier, which is you just look at the text of the assertion violation messages,
  • 1:03 - 1:06
    and you go ahead and make the assumption that distinct assertion violation messages
  • 1:06 - 1:10
    are caused by distinct bugs in the software under test, and that doesn't need to be true.
  • 1:10 - 1:13
    Not too long ago I reported something like six bugs in the LLVM compiler
  • 1:13 - 1:15
    that had completely distinct symptoms--
  • 1:15 - 1:18
    that is to say, completely distinct assertion violation messages--
  • 1:18 - 1:20
    but they were all traced back to a single error in the source code.
  • 1:20 - 1:22
    Of course that kind of thing can happen,
  • 1:22 - 1:25
    and that's especially common if the underlying bug in the software under test
  • 1:25 - 1:27
    is some sort of a memory safety violation.
  • 1:27 - 1:29
    That is to say, we have a C or C++ program
  • 1:29 - 1:31
    that starts corrupting it's own internal memory.
  • 1:31 - 1:34
    In that case, a single bug can map to any number of symptoms.
  • 1:34 - 1:36
    One possibility is that we have a single root cause
  • 1:36 - 1:38
    that is a defect in the software under test.
  • 1:38 - 1:41
    That can map multiple outputs that looks different.
  • 1:41 - 1:45
    This is one possibility, and this happens, although maybe not that often.
  • 1:45 - 1:49
    Another possibility is that we could have multiple defects that all cause the same symptom.
  • 1:49 - 1:51
    This is actually really common.
  • 1:51 - 1:54
    For example, in a big C or C++ program like a web browser
  • 1:54 - 1:56
    it's very common for multiple memory safety defects--
  • 1:56 - 1:59
    that is to say, multiple bugs that include things like buffer overruns
  • 1:59 - 2:01
    and null pointer references--
  • 2:01 - 2:03
    to all have the same symptom, which is segmentation fault.
  • 2:03 - 2:05
    That happens really commonly.
  • 2:05 - 2:08
    In contrast with these slightly pathological cases,
  • 2:08 - 2:11
    what we hope happens--and this is also relatively common--
  • 2:11 - 2:13
    is that a single defect maps to a single symptom.
  • 2:13 - 2:16
    By reporting a test case leading to that symptom, we can get that defect fixed.
  • 2:16 - 2:19
    So, in this case we have many defects,
  • 2:19 - 2:23
    all mapping to their own symptoms in the output space,
  • 2:23 - 2:25
    we can report these in parallel, and that's really nice.
  • 2:25 - 2:29
    Like I was saying, with large, complicated software that's evolving rapidly,
  • 2:29 - 2:31
    reporting bugs in parallel may be our only option,
  • 2:31 - 2:34
    if we want to actually get the number of defects in the software under control.
Tytuł:
Bug Triage - Software Testing
Opis:

09-03 Bug Triage

more » « less
Video Language:
English
Team:
Udacity
Projekt:
CS258: Software Testing
Duration:
02:38
Udacity Robot edited angielski subtitles for 09-03 Bug Triage
Udacity Robot edited angielski subtitles for 09-03 Bug Triage
Gundega added a translation

English subtitles

Revisions Compare revisions