English subtitles

← 03-05 Test Coverage

03-05 Test Coverage

Get Embed Code
1 Language

Showing Revision 2 created 07/16/2014 by Udacity Robot.

  1. Test coverage letâs us assign a score to a collection of test cases
  2. and so let's be a little bit more rigorous about it.
  3. So test coverage is really is a measure of the proportion of a program exercised during testing.
  4. So, for example, weâve just talked about measuring a number of functions
  5. out of the total number of functions or exercise by some test that we had.
  6. Whatâs good about test coverage is it gives us a score,
  7. gives us something objective that we can use to try figure out how well weâre doing.
  8. Additionally, when coverage is less than 100%, that is to say, as in our example,
  9. where we had failed to execute all of the functions in the software under test,
  10. we know what we need to do to get full coverage.
  11. We know what the functions are that we need to execute.
  12. Now, we simply need to construct test cases and execute those functions.
  13. So, these are the good things about test coverage.
  14. On the other hand, there are also some disadvantages.
  15. First of all, test coverage, because it is a white box metric that is derived
  16. from the source code for our system is not good at helping us find bugs of omission,
  17. that is to say bugs were simply left out something that we should have implemented.
  18. The second drawback is it could be really hard to know what a test coverage score less 100% means
  19. and in safety critical software development would sometimes done is requiring 100% test coverage
  20. of a certain coverage metrics and that sort of removes this problem,
  21. it means that we donât have to interpret scores less than 100% because weâre not allowed
  22. to ship our product until we get 100% test coverage.
  23. For a larger, more complex software systems where the standards are correct
  24. and not as high as they are for safety critical systems, thatâs often the case,
  25. but itâs difficult or impossible to achieve 100% test coverage.
  26. Leaving us with this problem weâre trying to figure out what that actually means about the software.
  27. The third disadvantage is even 100% coverage doesnât mean that all bugs are found
  28. and you could see that sort of easily by thinking about the example why weâre measuring
  29. our coverage by looking at the number of functions we executed.
  30. Just we executed some function, of course,
  31. it doesnât mean that we found all the bugs in that function.
  32. We may not have executed very much of it or may not have somehow
  33. found very many of the interesting behaviors inside that function.