YouTube

Got a YouTube account?

New: enable viewer-created translations and captions on your YouTube channel!

English subtitles

← Test Case Reduction - Software Testing

Get Embed Code
1 Language

Showing Revision 3 created 05/25/2016 by Udacity Robot.

  1. Test-case reduction is the process of taking some large input
  2. that triggers a failure and turning it into a small input.
  3. It's usually the case that many bugs can be triggered by small inputs.
  4. On the other hand, it's often the case that, for example, we discover a Firefox crash,
  5. and the web page that causes the Firefox crash might be giant. It's probably huge.
  6. We discover this bug in the wild, and statistically speaking, the pattern in this input
  7. that triggers the Firefox crash just probably isn't going to be found on some small web page.
  8. What we can do is, by hand, figure out what part of the input causes the test case.
  9. One thing we can do is eliminate part of the input.
  10. Sometimes we do this in a smart way.
  11. For example, we might just know that some part of it is unlikely to trigger a crash,
  12. or we might just chop some of it out blindly and see if the smaller input triggers the test case.
  13. If it doesn't, then we got back to our original test case and try again.
  14. If it does, we proceed. Now maybe it's this part.
  15. If we're really lucky, at the very end of this process, we'll end up with a really small test case.
  16. That's the thing we'd like to report to the people developing the software under test.
  17. Of course, even if we're not reporting bugs to someone else--
  18. that is to say, we're finding bugs in software that we wrote--
  19. it's still really nice to have a minimized test case,
  20. because these make it much easier to track down failures.
  21. Option one--manual reduction.
  22. This has been done by people debugging for probably
  23. about as long as computer science has been around.
  24. The second option is really cool. It's a really, really nice technique.
  25. It's called "delta debugging," and it automates this process.
  26. If you can write a program that can tell automatically if a particular input triggers a failure--
  27. that is to say, you load up the webpage in Firefox
  28. and you'll see if it crashes by looking at the exit code that it provides
  29. to the operating system.
  30. Then delta debugging is a framework that takes your script
  31. and takes the test input and automates this process in a loop,
  32. and this loop terminates when the delta debugger,
  33. which has a bunch of heuristics built in for eliminating parts of the input,
  34. it terminates when this delta debugger can't reduce the input anymore.
  35. What I don't want to do is going into this technique in a ton of detail,
  36. because the guy who developed the delta debugging is going
  37. to be teaching a Udacity class sometime soon,
  38. and it's probably going to be really interesting.
  39. What I hope is that I've intrigued you enough that you'll really serious consider taking his class.
  40. It's going to be about debugging. What I'd rather do now is show you an example.