English subtitles

← 03-11 Problems With Coverage Solution

03-11 Problems With Coverage Solution

Get Embed Code
1 Language

Showing Revision 2 created 07/16/2014 by Udacity Robot.

  1. So, here we are back with our broken formality test.
  2. And what I have done is I've changed the input a little bit so let's run these new inputs.
  3. And what we can see is the function has not made a couple of mistakes.
  4. For example, it is indicated that 9 is prime, and it is indicated that 25 is prime
  5. or of course neither of those numbers is prime.
  6. So, let's go back to the code and look the mistake. The algorithm here is generally a valid one.
  7. It's fine to first of all special case the even numbers,
  8. and it's fine then to loop between 3 and the square root of the number.
  9. So, the problem is that Python's range function
  10. does not loop through 3 and the square of the input number
  11. rather it loops between 3 and 1 less than the square of the input number.
  12. So, we can fix this, of course, easily by adding 1 to the square root
  13. causing it to actually test against the full range that it needs to test
  14. in order to have a successful formality test.
  15. So, let us run this code and see what happens.
  16. So, 6 is not prime, 7 is. None of the rest of these numbers are prime except for 29.
  17. So, the question is what happened here? Why did test coverage fail to identify the bug?
  18. And the answer is sort of simple--statement coverage is a rather crude metric
  19. that only checks whether each statement executes once.
  20. Each statement executes at least once that lets a lot of bugs slip through.
  21. For example, we can have a statement execute, but it computes the wrong numerical answer
  22. or what happened here a loop executed,
  23. but it executed for the wrong number of times computing the wrong result.
  24. The lesson here is we should not let complete coverage plus a number of successful test cases
  25. fool as into thinking that a piece of code of right.
  26. It's often the case that deeper analysis is necessary.