Return to Video

05-34 Strong Oracles

  • 0:00 - 0:04
    Strong oracles are extremely useful and you should always use one if you can possibly find one.
  • 0:04 - 0:06
    We're going to have a bunch of examples here.
  • 0:06 - 0:10
    Let's start off with one of the important ones which is having an alternate implementation
  • 0:10 - 0:14
    of the same specification, and if you think about it, this is what my random tester
  • 0:14 - 0:18
    for the bounded queue did and perhaps what yours also did, and what I mean by that is
  • 0:18 - 0:23
    my queue tester included a second implementation of the same queue abstraction.
  • 0:23 - 0:26
    This one was implemented with a Python list and this was the second
  • 0:26 - 0:28
    implementation of the same abstraction.
  • 0:28 - 0:31
    We could use to actually check that the queue that we're testing gave the right answers.
  • 0:31 - 0:33
    That's a very strong kind of a check.
  • 0:33 - 0:36
    Another example that we use in my compiler testing work--
  • 0:36 - 0:40
    we do what's called differential testing of compilers, and what that basically means is that if we have multiple
  • 0:40 - 0:45
    implementations of the same compiler specification--that is to say, for example, multiple C compilers--
  • 0:45 - 0:48
    we expect them to behave the same given equivalent inputs.
  • 0:48 - 0:51
    Another way that we might get an alternative implementation is simply looking of an older
  • 0:51 - 0:53
    version of the software that we're testing.
  • 0:53 - 0:57
    This is checking not necessarily whether the software is correct but just whether we've broken it.
  • 0:57 - 0:59
    And so remember for example that I said that in S.U.T probably could have tested
  • 0:59 - 1:03
    the Pentium floating-point unit against the 487 floating-point unit.
  • 1:03 - 1:08
    Another kind of old version oracle that tends to be extremely effective is after a refactoring change--
  • 1:08 - 1:12
    that is to say, the change of our source code base that isn't intended to have any functional effect--
  • 1:12 - 1:14
    we could use the version before and after refactoring.
  • 1:14 - 1:18
    We can do differential testing of the version before and after refactoring and in that way
  • 1:18 - 1:21
    try to get a pretty good idea that the refactoring did not actually break the code.
  • 1:21 - 1:25
    The best kind of alternate implementations you could have is a reference implementation.
  • 1:25 - 1:29
    That is to say, some sort of implementation of the specification, which you are after that you can trust.
  • 1:29 - 1:33
    For example, following to implement an extremely high performance Python compiler,
  • 1:33 - 1:35
    what I would do is I would use the regular C Python implementation
  • 1:35 -
    as the reference implementation, and that would be treated as correct.
Title:
05-34 Strong Oracles
Description:

05-34 Strong Oracles

more » « less
Team:
Udacity
Project:
CS258: Software Testing
Duration:
01:39
Udacity Robot edited English subtitles for Strong Oracles - Software Testing - Random Testing - Udacity
Amara Bot added a translation

English subtitles

Incomplete

Revisions Compare revisions