English subtitles

← 03-21 Mc Dc Coverage

03-21 Mc Dc Coverage

Get Embed Code
1 Language


Showing Revision 2 created 07/16/2014 by Udacity Robot.

  1. So, we're going to start off with the Python statement "if A or B and C:",
  2. so let's make sure to nail down the precedents so we don't have to remember that.
  3. If A is true, or else both B and C are true,
  4. then we're going to execute some code.
  5. And so now let's look what it takes to get full MC/DC coverage
  6. of this bit of code.
  7. The first thing we can see is that we're going to need to test each of the variables
  8. with both to their true and false values,
  9. because the conditions, that is to say, the conditions are A, B, and C here,
  10. need to take on all possible values.
  11. We can see that each of the conditions is going to need to be assigned both the true value
  12. and the false value during the test that we run.
  13. Now, the other part of MC/DC coverage--that is, does every condition independently affect
  14. the outcome of a decision--is going to be a little harder to deal with.
  15. Let's take a look.
  16. Let's first consider the input where A is true, B is false, and C is true.
  17. We don't even need to look at the B and C part of the expression,
  18. because we know that if A is true, then the entire condition succeeds.
  19. This maps to a true value.
  20. What we want to verify here is that we can come up with a test case where every condition
  21. independently affects that outcome of a decision.
  22. Since our top-level operator here is an or,
  23. let's see how we can make the whole thing come out false.
  24. While the B and C clause already came out to false,
  25. because if B is false, then the whole thing comes out to false.
  26. If we make A false, then the entire decision will come out to be false,
  27. and if we've changed only A and we haven't changed B and C,
  28. then we've shown that A independently affects the outcome.
  29. So, let's write another test case.
  30. Our second test case with A being input as false, B input as false, and C as true,
  31. leads the overall decision to come out as false.
  32. We've shown now that A independently affects the outcome.
  33. So let's try to do the same thing for B and C.
  34. If we want to continue trying to leave everything the same and only change the value of one variable
  35. in order to establish this independence condition,
  36. let's this time try flipping the value of B.
  37. We're going to have A being false, B being true, and C being true.
  38. If we look at the overall value of the decision, what now happens is that B and C evaluates to true,
  39. so it doesn't matter that A evaluates to false.
  40. Overall, decision evaluates to true this time.
  41. By flipping the value of only B we've satisfied this condition for the input B.
  42. That is to say we've shown that B independently affects the outcome,
  43. because when we change B, the overall value of the decision went from false to true.
  44. Now let's see if we can do the same thing for C.
  45. We're going to leave A and B the same, and we're going to pass in C as false.
  46. Now let's look what happens.
  47. B and C evaluates to false, and then also A is false, so the entire value of the boolean decision
  48. comes out to be false.
  49. By only changing C and by seeing that the overall decision changes value,
  50. we've now shown that C independently affects the outcome of the decision.
  51. So, what I believe we have here is a minimal--of if not minimal, at least fairly small--
  52. set of test cases that together gets 100% MC/DC coverage
  53. for this particular conditional statement in Python.
  54. You can see here that this isn't a particularly complicated conditional.
  55. We could've written one much more complicated,
  56. and if we had, we probably would've had a fairly hard time reasoning this stuff out by hand,
  57. and what we would've needed to do in that case is probably draw out a full truth table.
  58. So, let's look at the idea behind MC/DC coverage.
  59. Why would this be a good thing at all?
  60. What it's done is taken a statement that was really very easy to cover
  61. using either branch coverage or statement coverage.
  62. That is to say it's pretty easy to take this and make it either come out to be true or false overall
  63. on the force testing of the individual components of the boolean logic.
  64. Basically, the idea is that when we have complicated boolean expressions,
  65. they're truth tables become rather large.
  66. What that means is there's a lot of complexity hiding in those truth tables.
  67. When there's complexity, there are probably things we don't understand,
  68. and that means they're probably bugs.
  69. It turns out that the domain of interest for MC/DC coverage--
  70. that is to say embedded control systems that happen to be embedded in avionics systems
  71. end up having generally lots of complicated conditionals.
  72. It's definitely desirable when people's lives depend on the correctness
  73. of these complicated conditional expressions to force people to test them rather thoroughly.
  74. The other idea behind MC/DC coverage is that as part of establishing
  75. that every condition independently affects the outcome of a decision
  76. we're going to figure out when we have conditionals
  77. that don't independently affect the outcome of a decision.
  78. If you think about it, why would you have a conditional involved in a conditional expression
  79. which doesn't affect the outcome.
  80. What that basically means is there's a programming mistake, and it may be a harmless mistake.
  81. That is to say, the extra conditional being part of a decision
  82. may not affect the correctness of the overall system,
  83. but what it means is somebody didn't understand what was going on,
  84. and probably there's something that we need to look at more closely
  85. and probably even change.
  86. Another thing you can see, looking at MC/DC coverage,
  87. is that getting it on a very large piece of software is going to take an enormous amount of work.
  88. This is why it's a specialized coverage metric for the avionics industry
  89. where the critical parts of the codes end up being fairly small.
  90. That's MC/DC coverage, and we're not going to take a programming quiz on that,
  91. since first of all, as you saw, it gets to be a pain pretty quickly,
  92. and second we lack good tool support for MC/DC coverage in Python.