Return to Video

Test Case Reduction - Software Testing

  • 0:00 - 0:03
    Test-case reduction is the process of taking some large input
  • 0:03 - 0:06
    that triggers a failure and turning it into a small input.
  • 0:06 - 0:09
    It's usually the case that many bugs can be triggered by small inputs.
  • 0:09 - 0:13
    On the other hand, it's often the case that, for example, we discover a Firefox crash,
  • 0:13 - 0:16
    and the web page that causes the Firefox crash might be giant. It's probably huge.
  • 0:16 - 0:20
    We discover this bug in the wild, and statistically speaking, the pattern in this input
  • 0:20 - 0:24
    that triggers the Firefox crash just probably isn't going to be found on some small web page.
  • 0:24 - 0:28
    What we can do is, by hand, figure out what part of the input causes the test case.
  • 0:28 - 0:31
    One thing we can do is eliminate part of the input.
  • 0:31 - 0:34
    Sometimes we do this in a smart way.
  • 0:34 - 0:36
    For example, we might just know that some part of it is unlikely to trigger a crash,
  • 0:36 - 0:40
    or we might just chop some of it out blindly and see if the smaller input triggers the test case.
  • 0:40 - 0:44
    If it doesn't, then we got back to our original test case and try again.
  • 0:44 - 0:47
    If it does, we proceed. Now maybe it's this part.
  • 0:47 - 0:51
    If we're really lucky, at the very end of this process, we'll end up with a really small test case.
  • 0:51 - 0:55
    That's the thing we'd like to report to the people developing the software under test.
  • 0:55 - 0:57
    Of course, even if we're not reporting bugs to someone else--
  • 0:57 - 1:00
    that is to say, we're finding bugs in software that we wrote--
  • 1:00 - 1:01
    it's still really nice to have a minimized test case,
  • 1:01 - 1:04
    because these make it much easier to track down failures.
  • 1:04 - 1:06
    Option one--manual reduction.
  • 1:06 - 1:08
    This has been done by people debugging for probably
  • 1:08 - 1:10
    about as long as computer science has been around.
  • 1:10 - 1:13
    The second option is really cool. It's a really, really nice technique.
  • 1:13 - 1:17
    It's called "delta debugging," and it automates this process.
  • 1:17 - 1:22
    If you can write a program that can tell automatically if a particular input triggers a failure--
  • 1:22 - 1:24
    that is to say, you load up the webpage in Firefox
  • 1:24 - 1:26
    and you'll see if it crashes by looking at the exit code that it provides
  • 1:26 - 1:28
    to the operating system.
  • 1:28 - 1:31
    Then delta debugging is a framework that takes your script
  • 1:31 - 1:34
    and takes the test input and automates this process in a loop,
  • 1:34 - 1:37
    and this loop terminates when the delta debugger,
  • 1:37 - 1:40
    which has a bunch of heuristics built in for eliminating parts of the input,
  • 1:40 - 1:43
    it terminates when this delta debugger can't reduce the input anymore.
  • 1:43 - 1:46
    What I don't want to do is going into this technique in a ton of detail,
  • 1:46 - 1:49
    because the guy who developed the delta debugging is going
  • 1:49 - 1:51
    to be teaching a Udacity class sometime soon,
  • 1:51 - 1:53
    and it's probably going to be really interesting.
  • 1:53 - 1:57
    What I hope is that I've intrigued you enough that you'll really serious consider taking his class.
  • 1:57 - 2:01
    It's going to be about debugging. What I'd rather do now is show you an example.
Tytuł:
Test Case Reduction - Software Testing
Opis:

more » « less
Video Language:
English
Team:
Udacity
Projekt:
CS258: Software Testing
Duration:
02:01
Udacity Robot edited angielski subtitles for 09-05 Test Case Reduction
Udacity Robot edited angielski subtitles for 09-05 Test Case Reduction
Gundega added a translation

English subtitles

Revisions Compare revisions