1 00:00:25,699 --> 00:00:27,679 PAVAN SUDARSHAN: Hi. My name is Pavan. 2 00:00:27,679 --> 00:00:31,320 ANANDHA KRISHNAN: And I'm Anandha Krishnan. I'm 3 00:00:31,320 --> 00:00:35,980 also called Jake. Not Anandha. I know. We work 4 00:00:35,980 --> 00:00:37,390 at MavenHive technologies. 5 00:00:37,390 --> 00:00:40,449 P.S.: This is probably the only 6 00:00:40,449 --> 00:00:42,760 talk where there are two speakers, and we haven't 7 00:00:42,760 --> 00:00:45,260 rehearsed who says what, so we'll be stepping on 8 00:00:45,260 --> 00:00:47,750 each other's toes, so yeah. Bear with us. So 9 00:00:47,750 --> 00:00:50,260 yeah. Quick disclaimer: Most of what we are going 10 00:00:50,260 --> 00:00:52,589 to talk about is actually platform and language independent, 11 00:00:52,589 --> 00:00:55,510 or at least, in a sense, learned. But the 12 00:00:55,510 --> 00:00:58,369 reason we are talking here in a Ruby Conf 13 00:00:58,369 --> 00:01:01,180 is because of the Ruby community. We really think 14 00:01:01,180 --> 00:01:02,879 that a lot of things we are going to 15 00:01:02,879 --> 00:01:04,739 talk about resonates really well with the community, and 16 00:01:04,739 --> 00:01:07,290 we are hoping to, you know, drive a good 17 00:01:07,290 --> 00:01:10,890 discussion, conversation, whatever it is, from this audience. So 18 00:01:10,890 --> 00:01:13,340 that's really why we are talking here on this 19 00:01:13,340 --> 00:01:13,659 topic. 20 00:01:13,659 --> 00:01:15,350 A.K.: And a lot of the points that 21 00:01:15,350 --> 00:01:17,460 we're going to talk about kind of naturally apply 22 00:01:17,460 --> 00:01:22,390 to Rails and Ruby. And most of these we 23 00:01:22,390 --> 00:01:25,860 learned, and we sort of experience in our projects, 24 00:01:25,860 --> 00:01:29,490 which are mostly in Ruby and Rails, so. 25 00:01:29,490 --> 00:01:33,619 P.S.: Yeah, so let's start with condition. We have screwed 26 00:01:33,619 --> 00:01:36,289 up a lot in our careers. Right, between me 27 00:01:36,289 --> 00:01:38,740 and Jake, we have no idea how many mistakes 28 00:01:38,740 --> 00:01:41,219 we have made. And on those rare occassions, we 29 00:01:41,219 --> 00:01:43,130 have actually learned from it, or at least we 30 00:01:43,130 --> 00:01:46,359 would like to think so. So yeah, this talk 31 00:01:46,359 --> 00:01:52,789 is about one such mistake from which we learned, 32 00:01:52,789 --> 00:01:53,100 and yeah. 33 00:01:53,100 --> 00:01:54,899 A.K.: And, yes, I think most of 34 00:01:54,899 --> 00:01:57,439 it, we just put it up front, just based 35 00:01:57,439 --> 00:02:00,389 on projects, and you know, as we were talking 36 00:02:00,389 --> 00:02:03,520 about what happened with each of us, and things 37 00:02:03,520 --> 00:02:06,069 like that. So yeah, just trying to make it, 38 00:02:06,069 --> 00:02:09,008 you know, presentable and that stuff. But we- 39 00:02:09,008 --> 00:02:11,680 P.S.: Yeah, like, though we screwed up, we would like 40 00:02:11,680 --> 00:02:14,170 to believe that no employers or paranoid androids were 41 00:02:14,170 --> 00:02:17,360 hurt in the process of our mistakes, so yeah. 42 00:02:17,360 --> 00:02:21,230 OK, about three months back, so why pharamcist and 43 00:02:21,230 --> 00:02:23,930 doctors, right? So about three months about I was 44 00:02:23,930 --> 00:02:28,120 in this pharmacy buying diapers for my daughter, and 45 00:02:28,120 --> 00:02:30,090 in walks this guy - he just goes straight 46 00:02:30,090 --> 00:02:33,700 to the pharmacist and he's like, hey, can you 47 00:02:33,700 --> 00:02:36,909 give me something for a toothache? There was something 48 00:02:36,909 --> 00:02:40,489 very interesting and weird about this, and Jake and 49 00:02:40,489 --> 00:02:42,540 I, we carpool. So the next morning I was 50 00:02:42,540 --> 00:02:45,340 just telling Jake, and between the two of us, 51 00:02:45,340 --> 00:02:50,340 like, between the two of us, we realized that 52 00:02:50,340 --> 00:02:54,150 we have seen people ask for all sorts of 53 00:02:54,150 --> 00:02:56,829 medicines in a pharmacy. Right. Headaches, fever, like, like, 54 00:02:56,829 --> 00:02:59,640 true story, I even once saw this guy with 55 00:02:59,640 --> 00:03:04,519 a lot of cuts on his face from a 56 00:03:04,519 --> 00:03:06,939 knife, and yeah. Insane. Insane, right. So about, when 57 00:03:06,939 --> 00:03:08,930 we were talking about this, we thought there was 58 00:03:08,930 --> 00:03:11,390 something fundamentally wrong with this. Is there anyone here 59 00:03:11,390 --> 00:03:14,000 who thinks that it's totally OK for you to 60 00:03:14,000 --> 00:03:21,000 just walk up to a pharmacist and ask for 61 00:03:25,129 --> 00:03:28,120 a medicine? Oh yeah, cool. Yeah, so. Nice. OK. 62 00:03:28,120 --> 00:03:31,269 Hold that thought. Yeah, but like. So what we 63 00:03:31,269 --> 00:03:34,400 think, yes, a pharmacy potentially has a cure for 64 00:03:34,400 --> 00:03:37,390 pretty much any, most ailments that you could think 65 00:03:37,390 --> 00:03:41,120 of, and but the important thing is, though, you 66 00:03:41,120 --> 00:03:43,099 need to know what ailment you have. Right, there's 67 00:03:43,099 --> 00:03:47,260 that small implementation detail, right. And if it was 68 00:03:47,260 --> 00:03:48,620 that easy for you to just go to the 69 00:03:48,620 --> 00:03:51,840 right medicine and get it, this world would be 70 00:03:51,840 --> 00:03:56,730 filled with only pharmacists and not doctors, right. Yeah, 71 00:03:56,730 --> 00:03:59,329 so that's, so that's where the whole analogy starts, 72 00:03:59,329 --> 00:04:01,269 and then we'll get to how we connect to 73 00:04:01,269 --> 00:04:02,180 what we're going to talk about. 74 00:04:02,180 --> 00:04:05,810 A.K.: Yeah, and that's, that's where we, in a sort of thought 75 00:04:05,810 --> 00:04:09,159 that we use this metaphor to drive home that 76 00:04:09,159 --> 00:04:11,859 point. Of course, a lot of you might have 77 00:04:11,859 --> 00:04:15,269 your opinions about self-medication and the whole thing. So 78 00:04:15,269 --> 00:04:18,790 we'll stop it at this, and we will give 79 00:04:18,790 --> 00:04:21,269 us, our definition of what we think about these 80 00:04:21,269 --> 00:04:25,610 two sort of mindsets actually are and, you know. 81 00:04:25,610 --> 00:04:29,610 So starting off with like doctors, right. They don't 82 00:04:29,610 --> 00:04:32,580 treat, rarely, they don't try to treat the symptoms, 83 00:04:32,580 --> 00:04:34,710 it's about how you deal with them. So they 84 00:04:34,710 --> 00:04:36,970 go about just figuring out what the problem could 85 00:04:36,970 --> 00:04:40,120 be, you know, and then probably, you know, a 86 00:04:40,120 --> 00:04:43,330 lot of tests, make you run through a few 87 00:04:43,330 --> 00:04:46,250 tests or try and figure out what's what, if 88 00:04:46,250 --> 00:04:49,460 indeed it is the problem, and then try and, 89 00:04:49,460 --> 00:04:53,670 based on that, prescribe a treatment and, of course, 90 00:04:53,670 --> 00:04:55,660 make sure that you're OK at the end of 91 00:04:55,660 --> 00:04:58,080 it, right. The symptoms are gone. 92 00:04:58,080 --> 00:05:01,250 P.S.: And we didn't take a look through a medical textbook, so 93 00:05:01,250 --> 00:05:02,960 we don't know if this is right, but assuming 94 00:05:02,960 --> 00:05:06,060 it is, though, this is what worked for us, 95 00:05:06,060 --> 00:05:06,540 so. 96 00:05:06,540 --> 00:05:10,620 A.K.: And, again, in contrast, a pharmacist's job, 97 00:05:10,620 --> 00:05:14,360 we think very different, should be more about understanding 98 00:05:14,360 --> 00:05:18,600 the medicines, the medicines themselves. Probably even figure out 99 00:05:18,600 --> 00:05:21,000 what the disease is based on the medicine, right. 100 00:05:21,000 --> 00:05:24,850 But definitely they don't really think about, you know, 101 00:05:24,850 --> 00:05:26,790 what was the problem originally or what are we 102 00:05:26,790 --> 00:05:29,420 prescribing the treatment for. And they usually don't do 103 00:05:29,420 --> 00:05:31,220 it. Hopefully they don't do it. 104 00:05:31,220 --> 00:05:32,120 P.S.: Yeah. OK. 105 00:05:32,120 --> 00:05:33,880 So now with this context of what we mean 106 00:05:33,880 --> 00:05:37,560 by a doctor and a pharmacist and medicines and 107 00:05:37,560 --> 00:05:40,580 self-medication, all right. Let's get back to our mistake, 108 00:05:40,580 --> 00:05:44,310 which we want to talk about. Right. So our 109 00:05:44,310 --> 00:05:45,670 mistake that we want to talk about is a 110 00:05:45,670 --> 00:05:47,610 way we have dealt with, or rather we used 111 00:05:47,610 --> 00:05:50,780 to deal with bad symptoms on our code bases 112 00:05:50,780 --> 00:05:54,630 and on our projects, right. So you, a lot 113 00:05:54,630 --> 00:05:57,680 of times you see these in your code bases. 114 00:05:57,680 --> 00:06:01,130 There's the symptom, or there are some issues, right. 115 00:06:01,130 --> 00:06:04,360 So we obviously had a lot of those, in 116 00:06:04,360 --> 00:06:07,080 every single project we have worked on, and this 117 00:06:07,080 --> 00:06:09,310 is about how we dealt with that, right. 118 00:06:09,310 --> 00:06:12,740 A.K.: Let's start off with one very simple one, or 119 00:06:12,740 --> 00:06:14,690 at least the one which was the most easily 120 00:06:14,690 --> 00:06:15,440 expressible. 121 00:06:15,440 --> 00:06:18,930 P.S.: Yeah, as in, Tejas Dinkar, who has 122 00:06:18,930 --> 00:06:21,560 already been mentioned several times in different talks, he 123 00:06:21,560 --> 00:06:24,380 threatened us to throw off the stage if we 124 00:06:24,380 --> 00:06:28,410 took anything more than twenty-nine minutes, fifty-nine seconds. So 125 00:06:28,410 --> 00:06:32,390 we had to like really dumbify our, you know, 126 00:06:32,390 --> 00:06:34,890 anecdotes. But you learn, we have like a quick 127 00:06:34,890 --> 00:06:36,440 mention of different things which we would love to 128 00:06:36,440 --> 00:06:40,210 talk about offline, but yeah. So thanks Dejas. 129 00:06:40,210 --> 00:06:43,270 A.K.: So we'll get started on the first one. So 130 00:06:43,270 --> 00:06:45,800 this was a project where we had a familiar 131 00:06:45,800 --> 00:06:49,340 problem of regression bugs. We added new features, and 132 00:06:49,340 --> 00:06:53,460 that kept breaking things. So this is what we 133 00:06:53,460 --> 00:06:56,120 designed. We want this down. We want the number 134 00:06:56,120 --> 00:06:58,650 of bugs down from 10 to 5, you know. 135 00:06:58,650 --> 00:07:01,250 At that time it was, like, let's set up 136 00:07:01,250 --> 00:07:03,410 this goal, let's try and achieve this. What did 137 00:07:03,410 --> 00:07:05,410 we do? Oh, before that. Some facts about the 138 00:07:05,410 --> 00:07:09,470 project, right. This was not a project we started 139 00:07:09,470 --> 00:07:11,810 on from scratch, it was a lot of legacy 140 00:07:11,810 --> 00:07:13,280 code, a lot of code that we did not 141 00:07:13,280 --> 00:07:17,560 understand, and probably that's why we thought it was 142 00:07:17,560 --> 00:07:19,280 bad. And the test coverage was- 143 00:07:19,280 --> 00:07:21,910 P.S.: How many of you have taken over a code base from 144 00:07:21,910 --> 00:07:24,160 another team and thought the code base was awesome? 145 00:07:24,160 --> 00:07:29,110 Very small samples, so you realize what we mean 146 00:07:29,110 --> 00:07:32,320 when we thought it was bad. So. 147 00:07:32,320 --> 00:07:33,070 A.K.: Sure. 148 00:07:33,070 --> 00:07:36,200 So test-coverage was low, which was probably one of 149 00:07:36,200 --> 00:07:39,470 the reasons why people complained about the code base, 150 00:07:39,470 --> 00:07:41,990 of course. So what's your guess? 151 00:07:41,990 --> 00:07:46,090 P.S.: OK, so the problem we had was every time we checked 152 00:07:46,090 --> 00:07:48,640 in something, built a new feature, touch any part 153 00:07:48,640 --> 00:07:50,330 of the code base, we ended up breaking a 154 00:07:50,330 --> 00:07:52,280 whole bunch of other things. And we would not 155 00:07:52,280 --> 00:07:54,300 even know it right away, we would know it 156 00:07:54,300 --> 00:07:55,240 over a period of time. Right, so this was 157 00:07:55,240 --> 00:07:57,930 a regression problem. And given the facts, what would 158 00:07:57,930 --> 00:07:59,000 you probably have done? 159 00:07:59,000 --> 00:08:00,860 A.K.: I'll try and just 160 00:08:00,860 --> 00:08:03,910 go back again, and then hopefully forward - we're 161 00:08:03,910 --> 00:08:04,680 done [00:08:04]??. 162 00:08:04,680 --> 00:08:07,750 P.S.: Yeah, based on some facts. 163 00:08:07,750 --> 00:08:13,820 A.K.: Sure, so the low coverage was definitely a problem. 164 00:08:13,820 --> 00:08:17,040 We thought, I mean, everybody agreed that we need 165 00:08:17,040 --> 00:08:19,880 to start working on that, and you know, fix 166 00:08:19,880 --> 00:08:23,340 the coverage. So we went in there, put the 167 00:08:23,340 --> 00:08:26,150 coverage tool in place, you know. And then we 168 00:08:26,150 --> 00:08:28,170 decided we will write tests for every bug we 169 00:08:28,170 --> 00:08:32,610 caught. We got the coverage up, not very surprising, 170 00:08:32,610 --> 00:08:36,279 I mean, we didn't manage to- you know, improving 171 00:08:36,279 --> 00:08:37,200 the coverage- 172 00:08:37,200 --> 00:08:41,039 P.S.: Yeah like, so we spend like the whole time- 173 00:08:41,039 --> 00:08:42,539 A.K.: over a period of time. 174 00:08:42,539 --> 00:08:44,880 P.S.: OK, so, like so this was a problem, 175 00:08:44,880 --> 00:08:46,360 right. When you look at that, when you try 176 00:08:46,520 --> 00:08:48,380 to abstract the, into, like what our thought process 177 00:08:48,440 --> 00:08:56,540 was. It was something like this, right. Check. Yeah. 178 00:08:56,540 --> 00:08:58,620 So, we had a symptom. In this case it 179 00:08:58,620 --> 00:09:04,140 was low test-coverage, and - Jake - and we 180 00:09:04,140 --> 00:09:06,750 had, we decided on what metric and tool to 181 00:09:06,750 --> 00:09:09,700 use. In our case it was a simple line 182 00:09:09,700 --> 00:09:11,570 coverage, right and, archive?? [00:09:10], this was back in 183 00:09:11,570 --> 00:09:14,930 the days. And then we started solving that problem 184 00:09:14,930 --> 00:09:17,550 that we had, and then, you know, hopefully, for 185 00:09:17,550 --> 00:09:20,670 example, we were TDDing most of the new code 186 00:09:20,670 --> 00:09:23,970 that we wrote, so coverage was good on that, 187 00:09:23,970 --> 00:09:28,500 then start writing tests for things, which were, any 188 00:09:28,500 --> 00:09:29,980 part of the code base which we touched where 189 00:09:29,980 --> 00:09:31,720 there were no tests, we started adding tests there. 190 00:09:31,720 --> 00:09:33,900 You know, a bunch of different things. So basically, 191 00:09:33,900 --> 00:09:36,240 like, the idea was to improve the coverage and 192 00:09:36,240 --> 00:09:39,620 keep on writing, right. So cool, so. And what 193 00:09:39,620 --> 00:09:42,710 was the result? We ended up with, of course, 194 00:09:42,710 --> 00:09:45,390 drastically improving our test coverage, so we were in 195 00:09:45,390 --> 00:09:48,130 the late '90s for most the part of the 196 00:09:48,130 --> 00:09:50,630 code base, which was awesome. 197 00:09:50,630 --> 00:09:53,010 A.K.: A hundred, man, a hundred. 198 00:09:53,010 --> 00:09:55,890 P.S.: A hundred, yeah. Or, yeah, sure. 199 00:09:55,890 --> 00:10:00,070 Very good coverage. But things got only marginally better. 200 00:10:00,070 --> 00:10:02,200 At this point, this was when we realized that 201 00:10:02,200 --> 00:10:04,260 inspite of putting so much effort into actually improving 202 00:10:04,260 --> 00:10:07,000 our test coverage, our actual goal was to actual 203 00:10:07,000 --> 00:10:10,640 reduce the number of regression bugs. So we, we 204 00:10:10,640 --> 00:10:13,500 were still no better than what we started off, 205 00:10:13,500 --> 00:10:17,240 about two months back. So the developers were generally 206 00:10:17,240 --> 00:10:19,580 very happy, now they were doing a lot more 207 00:10:19,580 --> 00:10:22,360 TDD, they have a lot, they had manag- successfully 208 00:10:22,360 --> 00:10:24,600 convinced the manag- you know the product manager and 209 00:10:24,600 --> 00:10:27,590 the stake holders to spend more time on the 210 00:10:27,590 --> 00:10:29,600 tech deck?? [00:10:29] and things like that. They they 211 00:10:29,600 --> 00:10:31,800 also were happy. But the project manager was extremely 212 00:10:31,800 --> 00:10:34,430 frustrated, because in spite of spending so much effort, 213 00:10:34,430 --> 00:10:37,430 there was no real benefit from any of it, 214 00:10:37,430 --> 00:10:39,560 right. So it's like one of those very classic 215 00:10:39,560 --> 00:10:41,730 moments where you know, in Dilbert where, you know, 216 00:10:41,730 --> 00:10:45,029 the, OK, in Dilbert developers are never happy, but 217 00:10:45,029 --> 00:10:47,080 at least here we were happy and the project 218 00:10:47,080 --> 00:10:48,960 manager was sad, right. A.K.: And we weren't happy 219 00:10:48,960 --> 00:10:49,870 with the project managers as well. 220 00:10:49,870 --> 00:10:52,510 P.S.: Yeah, and eventually we feel there was something wrong, 221 00:10:52,510 --> 00:10:56,200 it's not like we take pleasure out of it. So, we 222 00:10:56,200 --> 00:10:58,630 think this is a very big mistake, where we 223 00:10:58,630 --> 00:11:01,380 spent almost two months of time without really realizing 224 00:11:01,380 --> 00:11:03,870 where we were going wrong, right. So this mistake 225 00:11:03,870 --> 00:11:07,490 and several mistakes across different projects which Dejas won't 226 00:11:07,490 --> 00:11:11,620 let us go into, ended up making us realize 227 00:11:11,620 --> 00:11:15,770 something very basic, right. And that's, this is basically 228 00:11:15,770 --> 00:11:17,000 what, this is what we were going to say. 229 00:11:17,000 --> 00:11:18,080 A.K.: OK, tell us a little bit. 230 00:11:18,080 --> 00:11:20,700 P.S.: If we had like a lightning talk, this is what 231 00:11:20,700 --> 00:11:22,050 we probably the only thing we would have put 232 00:11:22,050 --> 00:11:26,230 up and left. So never improve a metric. Solving 233 00:11:26,230 --> 00:11:29,850 a problem should automatically imrpve the metric that you're 234 00:11:29,850 --> 00:11:34,350 measuring, right. So the focus is on, is never 235 00:11:34,350 --> 00:11:36,700 on making a metric better. It's always about solving 236 00:11:36,700 --> 00:11:38,270 the problem. And, the metric- 237 00:11:38,270 --> 00:11:38,690 A.K.: This is like 238 00:11:38,690 --> 00:11:40,430 one of those, one of those things that is, 239 00:11:40,430 --> 00:11:41,620 it's very easily said and- 240 00:11:41,620 --> 00:11:42,080 P.S.: Yeah, and it- 241 00:11:42,080 --> 00:11:43,480 A.K.: You have to, at least for us, we 242 00:11:43,480 --> 00:11:45,070 always fell in that trap of- 243 00:11:45,070 --> 00:11:45,900 P.S.: Yeah, like 244 00:11:45,900 --> 00:11:48,480 it almost sounds like 'do the right thing,' but, 245 00:11:48,480 --> 00:11:51,090 yeah, like, it's very, it fits your common sense 246 00:11:51,090 --> 00:11:52,670 very well, but then when you're caught up in 247 00:11:52,670 --> 00:11:55,000 the daily, the day-to-day stuff in what you do 248 00:11:55,000 --> 00:11:57,029 in a project, it becomes very easy for you 249 00:11:57,029 --> 00:11:59,430 to miss the point here. So yeah, like this 250 00:11:59,430 --> 00:12:03,120 is really what, what is essence of what we 251 00:12:03,120 --> 00:12:05,779 are trying to say, right. 252 00:12:05,779 --> 00:12:08,190 A.K.: So, so what really happened here. 253 00:12:08,190 --> 00:12:09,050 Let's go a little bit more 254 00:12:09,050 --> 00:12:11,940 into what we were trying earlier and what we 255 00:12:11,940 --> 00:12:14,770 think we should have probably done. Instead of something 256 00:12:14,770 --> 00:12:18,800 like this, which, which actually ended up attacking the 257 00:12:18,800 --> 00:12:23,279 symptoms, or you know, targeting the symptom, we want 258 00:12:23,279 --> 00:12:25,430 to do something like this: There is a symptom, 259 00:12:25,430 --> 00:12:26,360 so just like always- 260 00:12:26,360 --> 00:12:27,480 P.S.: This is where the 261 00:12:27,480 --> 00:12:29,570 whole, like, the pharmacist and the doctor approach, yeah, 262 00:12:29,570 --> 00:12:32,279 like, it's a very long-shot metaphor, we agree, but- 263 00:12:32,279 --> 00:12:35,170 A.K.: The doctor thinking, which we hopefully want to 264 00:12:35,170 --> 00:12:39,800 do, is first is just try and take a 265 00:12:39,800 --> 00:12:41,339 guess at least at the problem, at least in 266 00:12:41,339 --> 00:12:42,810 our context, maybe not the doctor's. But in our 267 00:12:42,810 --> 00:12:46,570 context, take a quess at the problem, right. Think 268 00:12:46,570 --> 00:12:49,720 what it might be. Then that hopefully will tell 269 00:12:49,720 --> 00:12:52,230 you what you could do to probably solve the 270 00:12:52,230 --> 00:12:55,320 problem, solve, you know, that could be the solution 271 00:12:55,320 --> 00:12:59,230 which hope- will hopefully fix the problem, right. So 272 00:12:59,230 --> 00:13:01,740 this kind of very similar, we are iterating??[00:13:00] over 273 00:13:01,740 --> 00:13:05,380 this and hopefully, you know, improving on what we 274 00:13:05,380 --> 00:13:09,589 thought was the problem. And how do you know, 275 00:13:09,589 --> 00:13:12,420 then, that you know we are in fact improving 276 00:13:12,420 --> 00:13:13,870 on the problem? How do we know that is 277 00:13:13,870 --> 00:13:13,900 the problem? 278 00:13:13,900 --> 00:13:14,050 P.S.: Like the whole, how do you 279 00:13:14,050 --> 00:13:14,650 define them, right. So, how do we define them? 280 00:13:14,650 --> 00:13:17,900 A.K.: And that's where we think, that's where we 281 00:13:17,900 --> 00:13:21,420 think the metrics come in. Again, not metric, hopefully 282 00:13:21,420 --> 00:13:24,410 metrics, because that lets us measure the problem. It 283 00:13:24,410 --> 00:13:27,620 tells us at every point that, you know, you're 284 00:13:27,620 --> 00:13:30,440 doing better, you know, it's improving. And hopefully you 285 00:13:30,440 --> 00:13:34,620 also, when it gets done, right. So, yeah. So 286 00:13:34,620 --> 00:13:36,700 this is, this is probably the approach that we 287 00:13:36,700 --> 00:13:40,020 would like to try on, try from now on, 288 00:13:40,020 --> 00:13:43,860 also. And, right, there, you know, the problem may 289 00:13:43,860 --> 00:13:46,050 not be the, what is the one that we 290 00:13:46,050 --> 00:13:48,400 started out to fix. Like, like, probably in our 291 00:13:48,400 --> 00:13:50,610 previous case, you know, it, you should always be 292 00:13:50,610 --> 00:13:53,270 open to the idea that the problem will, what 293 00:13:53,270 --> 00:13:54,990 you thought was the problem was never the case, 294 00:13:54,990 --> 00:13:57,670 and it was not showing up. I mean, you 295 00:13:57,670 --> 00:14:00,029 were trying to, you were seeing the metrics improve, 296 00:14:00,029 --> 00:14:02,360 but then the symptom never went away, right. So 297 00:14:02,360 --> 00:14:04,040 be open to the notion that the problem could 298 00:14:04,040 --> 00:14:06,520 be different, in which case, the important thing is 299 00:14:06,520 --> 00:14:09,170 the solution is different and the metrics are different, 300 00:14:09,170 --> 00:14:10,380 right. So yeah. 301 00:14:10,380 --> 00:14:12,000 P.S.: Any guesses on what could 302 00:14:12,000 --> 00:14:15,240 have been the problem on that project? Where the 303 00:14:15,240 --> 00:14:20,460 regression bugs were written very high? It was duplication, 304 00:14:20,460 --> 00:14:23,180 as in there was, like, rampant duplication all over 305 00:14:23,180 --> 00:14:26,270 the place. And we would change something but forget 306 00:14:26,270 --> 00:14:28,410 to change the same thing in some other place. 307 00:14:28,410 --> 00:14:30,110 But, and because we didn't know about the code 308 00:14:30,110 --> 00:14:33,610 base, yeah. We were just blindly adding tests. And 309 00:14:33,610 --> 00:14:36,710 incrimentally going through different places, where each place where 310 00:14:36,710 --> 00:14:38,870 we found that there were no tests, we added 311 00:14:38,870 --> 00:14:40,470 a test, right. But that didn't really help us 312 00:14:40,470 --> 00:14:42,870 with actually solving the problem of duplication. Yeah. So- 313 00:14:42,870 --> 00:14:44,730 A.K.: The coverage number is something which is, it 314 00:14:44,730 --> 00:14:47,300 easily drives you to just keep adding the specs, 315 00:14:47,300 --> 00:14:50,210 and we will talk about, more about that soon. 316 00:14:50,210 --> 00:14:53,440 P.S.: Yeah, so, if you really think about it, 317 00:14:53,440 --> 00:14:55,980 of, basically what, at least, we would love to 318 00:14:55,980 --> 00:14:59,490 believe that we have stopped doing this, right. So 319 00:14:59,490 --> 00:15:02,490 Yogi mentioned this in the panel discussion yesterday, block-force 320 00:15:02,490 --> 00:15:05,779 driven decisions, right. So that was really what we 321 00:15:05,779 --> 00:15:07,870 were doing, essentially. Like, we were a bunch of 322 00:15:07,870 --> 00:15:10,899 kids on this project, who'd see a problem at 323 00:15:10,899 --> 00:15:14,220 the first, the, at the first, trigger we would 324 00:15:14,220 --> 00:15:17,399 just start crawling the web, start crawling block force, 325 00:15:17,399 --> 00:15:20,600 see different github projects, find a gem, install it, 326 00:15:20,600 --> 00:15:23,270 start, like, monitoring it, measuring it, try to improve 327 00:15:23,270 --> 00:15:25,510 it, you know. We don't really spending too much 328 00:15:25,510 --> 00:15:28,850 time into figuring out what was really the problem, 329 00:15:28,850 --> 00:15:35,850 right. And, especially, so in, OK - where does- 330 00:15:36,960 --> 00:15:40,399 Especially so in Rails projects, where, you know, or 331 00:15:40,399 --> 00:15:42,589 Ruby projects, where we believe that the number of 332 00:15:42,589 --> 00:15:46,800 gems which actually bundle best practices is actually very 333 00:15:46,800 --> 00:15:48,620 high, right. Here it's very easy for us to 334 00:15:48,620 --> 00:15:50,600 fall into the trap of, OK, just choose a 335 00:15:50,600 --> 00:15:53,360 gem, start using it, and then two months or 336 00:15:53,360 --> 00:15:54,690 three months down the way you have no idea 337 00:15:54,690 --> 00:15:56,589 where you used it in the first place. But 338 00:15:56,589 --> 00:15:58,580 it's just there in your process, right. Yeah. Like 339 00:15:58,580 --> 00:16:01,020 at least we used to find ourselves in that 340 00:16:01,020 --> 00:16:04,600 trap all the time. So yeah. This is basically 341 00:16:04,600 --> 00:16:08,200 what we stopped doing. OK, at this- 342 00:16:08,200 --> 00:16:08,820 A.K.: [indecipherable] 343 00:16:08,820 --> 00:16:10,740 P.S.: So this, this dude is basically Hari, 344 00:16:10,740 --> 00:16:13,680 he's sitting way over there. Yesterday we were doing 345 00:16:13,680 --> 00:16:16,600 a write-in??[00:16:14], and the only dude that, after this 346 00:16:16,600 --> 00:16:18,810 point it's fine, but it's getting monotonous, it's very 347 00:16:18,810 --> 00:16:21,760 black and white. And you need more images. And, 348 00:16:21,760 --> 00:16:24,690 like, Jake and I were really, we really think 349 00:16:24,690 --> 00:16:24,959 that- 350 00:16:24,959 --> 00:16:26,290 A.K.: That's our image, Hari! 351 00:16:26,290 --> 00:16:28,720 P.S.: We really think that we don't know how to add images 352 00:16:28,720 --> 00:16:31,290 to a presentation. And we were like, OK fine 353 00:16:31,290 --> 00:16:34,709 Hari, we'll just add your picture. And, yeah, so 354 00:16:34,709 --> 00:16:37,190 thanks for- and as you notice, we are very 355 00:16:37,190 --> 00:16:38,850 receptive of feedback, so. 356 00:16:38,850 --> 00:16:41,110 A.K.: He hasn't spoken yet, 357 00:16:41,110 --> 00:16:42,250 but his is full of interesting ones. 358 00:16:42,250 --> 00:16:44,959 P.S.: Like it would have been funny if like his was 359 00:16:44,959 --> 00:16:46,649 the presentation before we got the schedule, but yeah, 360 00:16:46,649 --> 00:16:51,080 anyway. He'll be talking next. So yeah, when you 361 00:16:51,080 --> 00:16:53,279 look at test coverage, right, how many of you 362 00:16:53,279 --> 00:16:59,010 think you understand test coverage very well? Well enough. 363 00:16:59,010 --> 00:17:03,970 OK, I mean, sure, yeah, like- 364 00:17:03,970 --> 00:17:07,510 A.K.: This was one thing which at least took us as a 365 00:17:07,510 --> 00:17:10,419 very obvious metrics thing, which we always get into, 366 00:17:10,419 --> 00:17:10,609 and- 367 00:17:10,609 --> 00:17:12,250 P.S.: When we were young and stupid as 368 00:17:12,250 --> 00:17:14,299 against now old and stupid, right, we were, we 369 00:17:14,299 --> 00:17:16,500 used to think oh, test coverage, what's that, it's 370 00:17:16,500 --> 00:17:18,799 just - meh. It's so easy, right. But then 371 00:17:18,799 --> 00:17:21,839 we realized even something so seemingly obvious had, like, 372 00:17:21,839 --> 00:17:25,059 so many different shades of details. And once you 373 00:17:25,059 --> 00:17:27,669 start understanding it and interpreting it in context is 374 00:17:27,669 --> 00:17:32,029 when you really understand the, the complexity of that, 375 00:17:32,029 --> 00:17:35,980 of that thing that you're measuring, right. Now think 376 00:17:35,980 --> 00:17:38,409 of all the things that people at Flipkart measure. 377 00:17:38,409 --> 00:17:40,820 I don't even know if they have a rational 378 00:17:40,820 --> 00:17:43,940 behind every one of it. I'm hoping they do, 379 00:17:43,940 --> 00:17:46,129 but you know like, it becomes very OK, we 380 00:17:46,129 --> 00:17:48,239 just need to monitor these ten things. Why? Why 381 00:17:48,239 --> 00:17:51,080 are you doing it? Right. So it should not 382 00:17:51,080 --> 00:17:52,940 be like a checklist at every project on start. 383 00:17:52,940 --> 00:17:55,519 You just start using it. So yeah, test coverage 384 00:17:55,519 --> 00:17:57,779 is definitely one thing that we found where, on 385 00:17:57,779 --> 00:17:59,059 start of every project we just set up a 386 00:17:59,059 --> 00:18:01,350 coverage tool. We just wanted to talk about some 387 00:18:01,350 --> 00:18:04,450 details on what we learned when doing that, so 388 00:18:04,450 --> 00:18:05,309 yeah. 389 00:18:05,309 --> 00:18:07,330 A.K.: So first we want to start off 390 00:18:07,330 --> 00:18:11,850 the obvious one. The measuring. So controller specs versus 391 00:18:11,850 --> 00:18:15,989 model specs coverage. I- I'm guessing it, does it 392 00:18:15,989 --> 00:18:20,989 ring any bells? I'm- so, so I'm, think of 393 00:18:20,989 --> 00:18:23,519 it this way, like, he has a question for 394 00:18:23,519 --> 00:18:27,320 you, right? You have, let's take a simple case. 395 00:18:27,320 --> 00:18:29,190 There is a single controller, you have a spec 396 00:18:29,190 --> 00:18:32,090 around it, there's a corresponding module, you have a 397 00:18:32,090 --> 00:18:34,279 spec around it, right. And then, you, with these 398 00:18:34,279 --> 00:18:35,330 two tests you run your coverage, right. 399 00:18:35,330 --> 00:18:36,649 P.S.: And you get some coverage from- 400 00:18:36,649 --> 00:18:39,059 A.K.: I'm guessing it's going to be a hundred percent. 401 00:18:39,059 --> 00:18:43,779 P.S.: Do you see a problem with this? Could there, rather, OK, 402 00:18:43,779 --> 00:18:49,320 could there be a problem with this? 403 00:18:49,320 --> 00:18:53,669 A.K.: What if you just removed the model spec? 404 00:18:53,669 --> 00:18:56,960 P.S.: What will the coverage for model be? Is there a 405 00:18:56,960 --> 00:19:01,570 chance that model's coverage is not zero? 406 00:19:01,570 --> 00:19:04,570 A.K.: Your controller spec is still gonna be loading the model. 407 00:19:04,570 --> 00:19:05,049 So your coverage- 408 00:19:05,049 --> 00:19:05,519 P.S.: Depends on- 409 00:19:05,519 --> 00:19:06,309 A.K.: -is still in question. 410 00:19:06,309 --> 00:19:08,139 P.S.: The answer is it depends, right, 411 00:19:08,139 --> 00:19:09,950 like, really depends on how you are testing your 412 00:19:09,950 --> 00:19:11,730 controller. But most of all things, what we have 413 00:19:11,730 --> 00:19:14,409 seen is, not every model is mocked out in 414 00:19:14,409 --> 00:19:17,169 the controller. Well, it's a totally different debate, whether 415 00:19:17,169 --> 00:19:19,980 should you mock your models or not, but if 416 00:19:19,980 --> 00:19:22,629 you are not modelin- or, mocking, your models are 417 00:19:22,629 --> 00:19:25,330 being loaded in your controller. So the controller spec, 418 00:19:25,330 --> 00:19:27,789 when it is tested for, like when the coverage 419 00:19:27,789 --> 00:19:31,289 is reported your models are being reported as well. 420 00:19:31,289 --> 00:19:33,759 A.K.: While, while the, or the point of, the 421 00:19:33,759 --> 00:19:35,600 point we are trying to make is, here, is 422 00:19:35,600 --> 00:19:37,919 not whether you should, how you should test your 423 00:19:37,919 --> 00:19:41,570 model specs and controller specs. What we do implore 424 00:19:41,570 --> 00:19:44,220 you to do is make sure that when you're 425 00:19:44,220 --> 00:19:45,989 doing, when you're looking at your coverage, you do 426 00:19:45,989 --> 00:19:48,350 have in mind your testing strategy, which is, am 427 00:19:48,350 --> 00:19:50,549 I actually mocking the model out or is it 428 00:19:50,549 --> 00:19:52,980 also getting wrote as a part of my controller 429 00:19:52,980 --> 00:19:54,879 spec? Is my controller spec also hitting the models, 430 00:19:54,879 --> 00:19:57,169 right? Think about these things when, when you're looking 431 00:19:57,169 --> 00:20:00,059 at these numbers, right. Or something that worked for 432 00:20:00,059 --> 00:20:02,179 us which we tried to do was we started 433 00:20:02,179 --> 00:20:05,730 monitoring the model specs coverage independently, and then started 434 00:20:05,730 --> 00:20:12,539 looking at the controller specs in light of the, 435 00:20:12,539 --> 00:20:14,470 in light of the model spec coverage. We wanted 436 00:20:14,470 --> 00:20:16,820 the model spec coverage to be high, because at 437 00:20:16,820 --> 00:20:19,649 least we wanted all, hopefully all our business logic 438 00:20:19,649 --> 00:20:22,590 was in the model specs, and you know, that's 439 00:20:22,590 --> 00:20:28,960 what we were keen on. Yes. Yeah, and then 440 00:20:28,960 --> 00:20:31,590 the next one, the line coverage itself, I think 441 00:20:31,590 --> 00:20:33,320 most commonly when we talk about coverage we just 442 00:20:33,320 --> 00:20:35,419 talk about line coverage. But then there is a 443 00:20:35,419 --> 00:20:38,080 bunch of other things as well, branch coverage, and 444 00:20:38,080 --> 00:20:39,830 then unique path coverage. 445 00:20:39,830 --> 00:20:40,710 P.S.: How many of you 446 00:20:40,710 --> 00:20:42,239 pay attention to branch coverages? 447 00:20:42,239 --> 00:20:44,730 A.K.: Or even monitor it? 448 00:20:44,730 --> 00:20:46,850 P.S.: How many of you don't think it's 449 00:20:46,850 --> 00:20:52,690 important? How many of you have no opinions? Cool. 450 00:20:52,690 --> 00:20:58,190 Yeah. I mean, sure. We have it on projects 451 00:20:58,190 --> 00:21:00,690 where it's been important, it's not been important, it's 452 00:21:00,690 --> 00:21:02,739 fine. But it's just that, you just need to 453 00:21:02,739 --> 00:21:05,830 know that it exists, and you need to train 454 00:21:05,830 --> 00:21:07,570 your data, right, so. 455 00:21:07,570 --> 00:21:08,590 A.K.: Just, hopefully it should 456 00:21:08,590 --> 00:21:11,809 not be a single metric. Something usually seems wrong 457 00:21:11,809 --> 00:21:13,019 if it is just gonna be about that one 458 00:21:13,019 --> 00:21:19,080 metric. Next one. So reporting. Yeah, this one is 459 00:21:19,080 --> 00:21:21,389 a bit tricky. What I usually don't like about 460 00:21:21,389 --> 00:21:24,480 coverage tools and these tools in general is they 461 00:21:24,480 --> 00:21:26,929 sometimes miss out this aspect of it. And they, 462 00:21:26,929 --> 00:21:28,489 in an attempt to try and be nice to 463 00:21:28,489 --> 00:21:30,820 you when you are very simple answered is try 464 00:21:30,820 --> 00:21:32,809 and give you a number which inherently makes it 465 00:21:32,809 --> 00:21:36,070 good or bad. There's nothing in between, and then 466 00:21:36,070 --> 00:21:38,960 the focus is lost. Like you either start liking 467 00:21:38,960 --> 00:21:41,340 it or you don't like it. You don't really 468 00:21:41,340 --> 00:21:46,149 think about what is there in between, right. Yeah. 469 00:21:46,149 --> 00:21:47,950 So the focus on, focus on some of the 470 00:21:47,950 --> 00:21:49,499 aspects of where is the coverage, you know, what 471 00:21:49,499 --> 00:21:51,909 does it mean, right. One thing that worked for 472 00:21:51,909 --> 00:21:55,109 us was code climate in the region projects. I 473 00:21:55,109 --> 00:21:56,989 really like it because they put a lot of 474 00:21:56,989 --> 00:21:59,700 focus into the presentation aspect of it. Not just 475 00:21:59,700 --> 00:22:03,179 the collection aspect of it. It really, you know, 476 00:22:03,179 --> 00:22:06,929 takes you down to the, to the code, which 477 00:22:06,929 --> 00:22:09,649 is missing the specs. Of course, they do also 478 00:22:09,649 --> 00:22:11,940 other metrics like code quality, which I really like, 479 00:22:11,940 --> 00:22:14,489 by the way. They have some notification things like 480 00:22:14,489 --> 00:22:17,109 that, like, on, like Lexus a lot, whenever this 481 00:22:17,109 --> 00:22:19,169 climate goes in, poof the coverage goes down or 482 00:22:19,169 --> 00:22:21,019 something like that. [00:22:19] It doesn't break the builder 483 00:22:21,019 --> 00:22:23,600 part, it doesn't break the build, but it lets 484 00:22:23,600 --> 00:22:25,080 you know, and then you can deal with it 485 00:22:25,080 --> 00:22:27,109 if you think it is important to deal with. 486 00:22:27,109 --> 00:22:30,389 P.S.: OK, speaking of breaking build, how many of 487 00:22:30,389 --> 00:22:35,859 you know what racheting, in builds? OK. So the 488 00:22:35,859 --> 00:22:37,999 idea of racheting is basically you will never leave 489 00:22:37,999 --> 00:22:41,100 your code base worse than what it already was, 490 00:22:41,100 --> 00:22:45,059 right. So every commit basically makes sure, even if 491 00:22:45,059 --> 00:22:46,759 it doesn't do any good, it doesn't do any 492 00:22:46,759 --> 00:22:48,779 bad to your code base. So for example, if 493 00:22:48,779 --> 00:22:51,859 you're, your current code coverage is at 70%, and 494 00:22:51,859 --> 00:22:53,840 if this check-in makes it 69%, it will break 495 00:22:53,840 --> 00:22:56,309 the build. Even though there's nothing functionally wrong with 496 00:22:56,309 --> 00:23:00,169 it, you know, it's bad, right. We really think 497 00:23:00,169 --> 00:23:02,600 it's a double-edged sword. I will, this is one 498 00:23:02,600 --> 00:23:05,249 of those things which I have a, in theory 499 00:23:05,249 --> 00:23:08,629 sounds very, very good, and direct, but in practice, 500 00:23:08,629 --> 00:23:10,950 what it typically ends up doing is people end 501 00:23:10,950 --> 00:23:14,539 up fretting the actual metric, and never about what 502 00:23:14,539 --> 00:23:17,570 the problem is, right. Becau- this exactly does what 503 00:23:17,570 --> 00:23:20,450 we said in the previous slide, which is coverage 504 00:23:20,450 --> 00:23:24,029 is never red or green, right. Sometimes you are 505 00:23:24,029 --> 00:23:26,359 OK with taking this hit because you want to 506 00:23:26,359 --> 00:23:28,859 do something. I mean there are all, there are 507 00:23:28,859 --> 00:23:33,720 so many reasons why which, in practic- in reality 508 00:23:33,720 --> 00:23:35,850 you may have to do some bad things, and 509 00:23:35,850 --> 00:23:37,509 eventually have to pay for it, but it's OK, 510 00:23:37,509 --> 00:23:40,690 it's a conscious decision, right. But racheting invariably stops 511 00:23:40,690 --> 00:23:44,759 that. It makes it very, you know, black and 512 00:23:44,759 --> 00:23:45,470 white, right. 513 00:23:45,470 --> 00:23:47,309 A.K.: Difficult for you to proceed at 514 00:23:47,309 --> 00:23:47,869 that very moment. 515 00:23:47,869 --> 00:23:49,369 P.S.: Yeah and it has a 516 00:23:49,369 --> 00:23:53,379 more behavioral impact on the team, which is your, 517 00:23:53,379 --> 00:23:56,850 your team members start hating either the racheting, or 518 00:23:56,850 --> 00:24:00,259 they start hating the metric, or you know, we 519 00:24:00,259 --> 00:24:03,799 had this one person who did not commit for 520 00:24:03,799 --> 00:24:06,129 four days because they thought they did not have 521 00:24:06,129 --> 00:24:09,299 enough test coverage. And, like, when we breached the 522 00:24:09,299 --> 00:24:12,350 whole, OK, frequent check-ins, call check-in, and this person 523 00:24:12,350 --> 00:24:14,389 was actually scared because they were about, they would 524 00:24:14,389 --> 00:24:17,600 break the build, is actually a, very, very bad 525 00:24:17,600 --> 00:24:20,729 signal, of, a bad sign from your team, right. 526 00:24:20,729 --> 00:24:22,690 Now well, you can always argue, OK, this person 527 00:24:22,690 --> 00:24:25,139 totally missed the point, we can say a bunch 528 00:24:25,139 --> 00:24:27,499 of things, right, but this is the reality. So 529 00:24:27,499 --> 00:24:29,739 we think that it's a good idea, so that 530 00:24:29,739 --> 00:24:32,039 you might want to do it at certain points 531 00:24:32,039 --> 00:24:35,229 in time. But yeah, be very, very careful about 532 00:24:35,229 --> 00:24:37,729 the freakanomic implication that it has on your team, 533 00:24:37,729 --> 00:24:38,080 right, always keep that in mind. 534 00:24:38,080 --> 00:24:40,419 A.K.: And then 535 00:24:40,419 --> 00:24:43,269 the very other popular thing, which is, you just 536 00:24:43,269 --> 00:24:46,109 write that test to bump up the coverage, if 537 00:24:46,109 --> 00:24:46,679 you're not just- 538 00:24:46,679 --> 00:24:47,309 P.S.: Yeah, so there was, there 539 00:24:47,309 --> 00:24:49,769 was just one more philosophy of full-on coverage?? [00:24:50], 540 00:24:49,769 --> 00:24:52,429 which is saying, OK, you just checked-in code, but 541 00:24:52,429 --> 00:24:54,249 you don't have a test for this, but that's 542 00:24:54,249 --> 00:24:56,669 OK. There's this other class which does not have 543 00:24:56,669 --> 00:24:58,669 a test for it, which is easy, so let's 544 00:24:58,669 --> 00:25:01,479 start this there and keep my coverage maintained right 545 00:25:01,479 --> 00:25:04,049 there. There's so many, like, weird things people end 546 00:25:04,049 --> 00:25:06,389 up doing, just because they are now worried about 547 00:25:06,389 --> 00:25:09,119 coverage and not really worried about what it means 548 00:25:09,119 --> 00:25:11,470 to- you know, what it means with what they 549 00:25:11,470 --> 00:25:12,419 are doing, right. So it's- 550 00:25:12,419 --> 00:25:14,759 is done with the best of intentions, but then 551 00:25:14,759 --> 00:25:16,749 you get really, really bad, cause, like- 552 00:25:16,749 --> 00:25:17,320 P.S.: Yeah. 553 00:25:17,320 --> 00:25:19,879 So, OK, how, how did we go, I mean, 554 00:25:19,879 --> 00:25:22,440 how do we now improve, like, so we still 555 00:25:22,440 --> 00:25:25,639 take on a lot of projects, customer, we are, 556 00:25:25,639 --> 00:25:27,600 we are mainly now in consulting, so we do 557 00:25:27,600 --> 00:25:29,889 end up taking worker code bases, right. So what 558 00:25:29,889 --> 00:25:32,389 do we do to improve coverage if we have 559 00:25:32,389 --> 00:25:34,690 a bad one? One thing we realized is adding 560 00:25:34,690 --> 00:25:37,950 unit tests to classes, existing classes, could be a 561 00:25:37,950 --> 00:25:41,200 very dangerous thing, right. What that essentially means, is 562 00:25:41,200 --> 00:25:43,369 you have, you know, you are cementing the current 563 00:25:43,369 --> 00:25:46,379 design. Now I won't say, like prejudiously, it might 564 00:25:46,379 --> 00:25:48,639 be bad to say not a good design, right. 565 00:25:48,639 --> 00:25:50,989 But you are cementing it, right. If ever you 566 00:25:50,989 --> 00:25:54,470 want to cement something, cement features. Cement functionality, right. 567 00:25:54,470 --> 00:25:56,529 Which means you might want to write like a 568 00:25:56,529 --> 00:26:00,059 much higher level test of, and, you know, ensure 569 00:26:00,059 --> 00:26:01,799 that the functionality is the same, so that you 570 00:26:01,799 --> 00:26:04,239 can go and refactor later. On this one project, 571 00:26:04,239 --> 00:26:07,009 which Yogi, me, and Jake worked together on in 572 00:26:07,009 --> 00:26:11,470 ThoughtWorks, it worked beautifully for us, where we were 573 00:26:11,470 --> 00:26:15,879 strangling ?? to Hibernate [00:26:14]. Which meant that all 574 00:26:15,879 --> 00:26:18,840 that entire unit tests and database level tests were 575 00:26:18,840 --> 00:26:21,960 completely invalidated, they were useless, right. And because of 576 00:26:21,960 --> 00:26:24,309 the way, and because we wanted to turn on 577 00:26:24,309 --> 00:26:26,470 to caches?? [00:26:24], transactions changed. Like that meant the 578 00:26:26,470 --> 00:26:30,100 whole app was rendered useless from a testing point 579 00:26:30,100 --> 00:26:31,359 of view, right, from a safety net point of 580 00:26:31,359 --> 00:26:34,979 view. But what came to our help was controller 581 00:26:34,979 --> 00:26:38,289 and beyond level tests. So our biggest, we had 582 00:26:38,289 --> 00:26:40,929 such good coverage there, that we went in and 583 00:26:40,929 --> 00:26:42,989 we just modified a whole bunch of things, and 584 00:26:42,989 --> 00:26:45,399 we like started deleting tests, you know. You get 585 00:26:45,399 --> 00:26:48,289 a lot more flexibility and freedom inside your code 586 00:26:48,289 --> 00:26:49,840 base when you know that you're not breaking any 587 00:26:49,840 --> 00:26:53,210 functionality. So yeah like it's a really, really, like, 588 00:26:53,210 --> 00:26:55,570 good thing, so definitely think about it when you're 589 00:26:55,570 --> 00:26:57,229 inheriting a legacy codebase. 590 00:26:57,229 --> 00:26:58,669 A.K.: Unit tests, unit tests 591 00:26:58,669 --> 00:27:01,279 are great, but do keep in mind that they 592 00:27:01,279 --> 00:27:03,029 might also come in the way of- 593 00:27:03,029 --> 00:27:03,619 P.S.: Changes. 594 00:27:03,619 --> 00:27:04,489 A.K.: Refactoring, and changes. 595 00:27:04,489 --> 00:27:04,909 P.S.: Yeah. 596 00:27:04,909 --> 00:27:05,899 A.K.: Big, big refactoring- 597 00:27:05,899 --> 00:27:07,739 P.S.: So, OK. What - the whole measurement 598 00:27:07,739 --> 00:27:11,399 reporting, racheting, improvement, all of this is basically saying, 599 00:27:11,399 --> 00:27:13,279 always keep the problem you're solving in mind, right. 600 00:27:13,279 --> 00:27:15,739 Rachet don't rachet - how do you decide? Well, 601 00:27:15,739 --> 00:27:17,950 is it helping you to achieve your goal, or 602 00:27:17,950 --> 00:27:20,460 the problem you have at hand? Sure, you know, 603 00:27:20,460 --> 00:27:26,499 so do it. Otherwise don't do it, right. It's- 604 00:27:26,499 --> 00:27:27,249 Yeah. 605 00:27:27,249 --> 00:27:28,659 A.K.: I think- 606 00:27:28,659 --> 00:27:29,999 P.S.: So, the second- 607 00:27:29,999 --> 00:27:30,669 A.K.: Let's- 608 00:27:30,669 --> 00:27:33,350 P.S.: -anecdote- I'll just be, quickly talk about- 609 00:27:33,350 --> 00:27:35,359 A.K.: We have five minutes, so- 610 00:27:35,359 --> 00:27:36,359 P.S.: Yeah, like, 611 00:27:36,359 --> 00:27:37,369 the second anecdote was basically, OK, we had this 612 00:27:37,369 --> 00:27:42,489 server which was under a very heavy load. And, 613 00:27:42,489 --> 00:27:48,629 like thousands of requests, a minute, and only about 614 00:27:48,629 --> 00:27:52,320 5% of those requests, in very seemingly arbitrary periods 615 00:27:52,320 --> 00:27:56,340 of times, and arbirtrary controllers and actions, would pause, 616 00:27:56,340 --> 00:27:59,409 and it would take a very long time, right. 617 00:27:59,409 --> 00:28:01,899 So the- this problem was way more technical. It 618 00:28:01,899 --> 00:28:04,169 had nothing to do with the behavior or, you 619 00:28:04,169 --> 00:28:09,440 know, like, or practices part of, or side of 620 00:28:09,440 --> 00:28:11,460 things. It was pure technical problem. And goal was 621 00:28:11,460 --> 00:28:13,059 for us to find the root cause of this 622 00:28:13,059 --> 00:28:17,200 unpredictable behavior and fix it, right. And, yeah, like 623 00:28:17,200 --> 00:28:20,340 we were, like we can definitely talk about how 624 00:28:20,340 --> 00:28:22,499 we went into, like a lot of different solving, 625 00:28:22,499 --> 00:28:24,549 a lot of different symptoms, at one point even 626 00:28:24,549 --> 00:28:28,159 suspecting JRuby. You know, like, so it's very, like, 627 00:28:28,159 --> 00:28:30,580 it becomes very hard for you to figure out, 628 00:28:30,580 --> 00:28:32,470 OK, what is a problem, what is a symptom, 629 00:28:32,470 --> 00:28:35,159 and like, go very methodical about it, right. So 630 00:28:35,159 --> 00:28:38,009 that's what this, this problem was going to be 631 00:28:38,009 --> 00:28:38,999 about. But let's take this offline. At this point 632 00:28:38,999 --> 00:28:39,029 we can- 633 00:28:39,029 --> 00:28:39,139 A.K.: I mean, before that, yeah, let's 634 00:28:39,139 --> 00:28:43,159 get out some questions. 635 00:28:43,159 --> 00:28:45,839 P.S.: Yeah, let's take some 636 00:28:45,840 --> 00:28:47,720 questions if you have any. 637 00:28:47,720 --> 00:28:48,920 A.K.: Cool. 638 00:28:48,920 --> 00:28:50,520 P.S.: Cool, thanks. 639 00:28:50,520 --> 00:28:51,720 A.K.: Thanks. 640 00:28:51,720 --> 00:28:53,180 P.S.: Thanks guys.