English subtitles

← Fuzzing - Software Testing

Get Embed Code
1 Language

Showing Revision 3 created 05/25/2016 by Udacity Robot.

  1. Well, I'd like to talk a little bit about how random testing differs from fuzzing
  2. and the short answer is they're the same thing.
  3. The long answer is going to require a bit of explanation.
  4. So let's go back in time to 1990 when a professor called Bart Miller and his students
  5. published a paper called An Empirical Study of the Reliability of Unix Utilities.
  6. And so what they did as part of the fuzzing effort was provide a completely random data
  7. to a bunch of Unix command-line utilities.
  8. These were things like editors, terminal programs, text processing utilities,
  9. and other similar Unix tools that you can basically think of as being tools predating the era of
  10. graphical use or interfaces on Unix systems.
  11. And what they found is using this incredibly simple technique, that is doing random testing
  12. without worrying at all about the input validity problem they were able to crash
  13. a quarter to a third of these utilities on basically any version of Unix that they tested.
  14. And so what you have here is a pretty strong result.
  15. They were able to crash lots of applications with minimal effort.
  16. What that means is that the quality of the input validation done by these sorts of programs
  17. at the time was really rather bad.
  18. A few years later in 1995, the same group repeated the effort and wrote another paper about this.
  19. This time they not only tested the same kind of utilities that they tested 5 years earlier
  20. but they extended the work to testing network applications and GUI applications
  21. and basically they got very similar results.
  22. Now, in another 5 years in 2000, the same people did another study
  23. and this time they fuzzed Windows applications.
  24. And what they found was basically more of the same.
  25. They can crash most of the applications that they tested.
  26. And then finally in 2006, the most recent installment of a fuzzing study by this group was published.
  27. This time they attacked Mac OS X.
  28. And this time they found something a little bit different.
  29. The command-line utilities on Max OS X would hardly crash.
  30. They found a much lower rate of crashes than they have found earlier.
  31. But on the other hand, of the 30 GUI apps that they tested, 22 could be crashed.
  32. It's worth mentioning that as this group evolved their fuzzing work,
  33. they kept having to write new tools.
  34. For example, to fuzz the Windows applications they had to generate Windows events
  35. to GUI applications and they had to do something similar for Mac OS
  36. and previously for X Windows applications.
  37. So they had to keep evolving their tools but the input generation methodology that they used,
  38. that is to say basically generating random garbage and not really worrying about the
  39. input validity problem remained the same across all of these studies.
  40. So now what I've covered so far was this particular random testing effort by this one research group.
  41. But something interesting happened I believe sometime around 2000 or a little after
  42. is the term fuzzing took on another use.