WEBVTT 00:00:25.699 --> 00:00:27.679 PAVAN SUDARSHAN: Hi. My name is Pavan. 00:00:27.679 --> 00:00:31.320 ANANDHA KRISHNAN: And I'm Anandha Krishnan. I'm 00:00:31.320 --> 00:00:35.980 also called Jake. Not Anandha. I know. We work 00:00:35.980 --> 00:00:37.390 at MavenHive technologies. 00:00:37.390 --> 00:00:40.449 P.S.: This is probably the only 00:00:40.449 --> 00:00:42.760 talk where there are two speakers, and we haven't 00:00:42.760 --> 00:00:45.260 rehearsed who says what, so we'll be stepping on 00:00:45.260 --> 00:00:47.750 each other's toes, so yeah. Bear with us. So 00:00:47.750 --> 00:00:50.260 yeah. Quick disclaimer: Most of what we are going 00:00:50.260 --> 00:00:52.589 to talk about is actually platform and language independent, 00:00:52.589 --> 00:00:55.510 or at least, in a sense, learned. But the 00:00:55.510 --> 00:00:58.369 reason we are talking here in a Ruby Conf 00:00:58.369 --> 00:01:01.180 is because of the Ruby community. We really think 00:01:01.180 --> 00:01:02.879 that a lot of things we are going to 00:01:02.879 --> 00:01:04.739 talk about resonates really well with the community, and 00:01:04.739 --> 00:01:07.290 we are hoping to, you know, drive a good 00:01:07.290 --> 00:01:10.890 discussion, conversation, whatever it is, from this audience. So 00:01:10.890 --> 00:01:13.340 that's really why we are talking here on this 00:01:13.340 --> 00:01:13.659 topic. 00:01:13.659 --> 00:01:15.350 A.K.: And a lot of the points that 00:01:15.350 --> 00:01:17.460 we're going to talk about kind of naturally apply 00:01:17.460 --> 00:01:22.390 to Rails and Ruby. And most of these we 00:01:22.390 --> 00:01:25.860 learned, and we sort of experience in our projects, 00:01:25.860 --> 00:01:29.490 which are mostly in Ruby and Rails, so. 00:01:29.490 --> 00:01:33.619 P.S.: Yeah, so let's start with condition. We have screwed 00:01:33.619 --> 00:01:36.289 up a lot in our careers. Right, between me 00:01:36.289 --> 00:01:38.740 and Jake, we have no idea how many mistakes 00:01:38.740 --> 00:01:41.219 we have made. And on those rare occassions, we 00:01:41.219 --> 00:01:43.130 have actually learned from it, or at least we 00:01:43.130 --> 00:01:46.359 would like to think so. So yeah, this talk 00:01:46.359 --> 00:01:52.789 is about one such mistake from which we learned, 00:01:52.789 --> 00:01:53.100 and yeah. 00:01:53.100 --> 00:01:54.899 A.K.: And, yes, I think most of 00:01:54.899 --> 00:01:57.439 it, we just put it up front, just based 00:01:57.439 --> 00:02:00.389 on projects, and you know, as we were talking 00:02:00.389 --> 00:02:03.520 about what happened with each of us, and things 00:02:03.520 --> 00:02:06.069 like that. So yeah, just trying to make it, 00:02:06.069 --> 00:02:09.008 you know, presentable and that stuff. But we- 00:02:09.008 --> 00:02:11.680 P.S.: Yeah, like, though we screwed up, we would like 00:02:11.680 --> 00:02:14.170 to believe that no employers or paranoid androids were 00:02:14.170 --> 00:02:17.360 hurt in the process of our mistakes, so yeah. 00:02:17.360 --> 00:02:21.230 OK, about three months back, so why pharamcist and 00:02:21.230 --> 00:02:23.930 doctors, right? So about three months about I was 00:02:23.930 --> 00:02:28.120 in this pharmacy buying diapers for my daughter, and 00:02:28.120 --> 00:02:30.090 in walks this guy - he just goes straight 00:02:30.090 --> 00:02:33.700 to the pharmacist and he's like, hey, can you 00:02:33.700 --> 00:02:36.909 give me something for a toothache? There was something 00:02:36.909 --> 00:02:40.489 very interesting and weird about this, and Jake and 00:02:40.489 --> 00:02:42.540 I, we carpool. So the next morning I was 00:02:42.540 --> 00:02:45.340 just telling Jake, and between the two of us, 00:02:45.340 --> 00:02:50.340 like, between the two of us, we realized that 00:02:50.340 --> 00:02:54.150 we have seen people ask for all sorts of 00:02:54.150 --> 00:02:56.829 medicines in a pharmacy. Right. Headaches, fever, like, like, 00:02:56.829 --> 00:02:59.640 true story, I even once saw this guy with 00:02:59.640 --> 00:03:04.519 a lot of cuts on his face from a 00:03:04.519 --> 00:03:06.939 knife, and yeah. Insane. Insane, right. So about, when 00:03:06.939 --> 00:03:08.930 we were talking about this, we thought there was 00:03:08.930 --> 00:03:11.390 something fundamentally wrong with this. Is there anyone here 00:03:11.390 --> 00:03:14.000 who thinks that it's totally OK for you to 00:03:14.000 --> 00:03:21.000 just walk up to a pharmacist and ask for 00:03:25.129 --> 00:03:28.120 a medicine? Oh yeah, cool. Yeah, so. Nice. OK. 00:03:28.120 --> 00:03:31.269 Hold that thought. Yeah, but like. So what we 00:03:31.269 --> 00:03:34.400 think, yes, a pharmacy potentially has a cure for 00:03:34.400 --> 00:03:37.390 pretty much any, most ailments that you could think 00:03:37.390 --> 00:03:41.120 of, and but the important thing is, though, you 00:03:41.120 --> 00:03:43.099 need to know what ailment you have. Right, there's 00:03:43.099 --> 00:03:47.260 that small implementation detail, right. And if it was 00:03:47.260 --> 00:03:48.620 that easy for you to just go to the 00:03:48.620 --> 00:03:51.840 right medicine and get it, this world would be 00:03:51.840 --> 00:03:56.730 filled with only pharmacists and not doctors, right. Yeah, 00:03:56.730 --> 00:03:59.329 so that's, so that's where the whole analogy starts, 00:03:59.329 --> 00:04:01.269 and then we'll get to how we connect to 00:04:01.269 --> 00:04:02.180 what we're going to talk about. 00:04:02.180 --> 00:04:05.810 A.K.: Yeah, and that's, that's where we, in a sort of thought 00:04:05.810 --> 00:04:09.159 that we use this metaphor to drive home that 00:04:09.159 --> 00:04:11.859 point. Of course, a lot of you might have 00:04:11.859 --> 00:04:15.269 your opinions about self-medication and the whole thing. So 00:04:15.269 --> 00:04:18.790 we'll stop it at this, and we will give 00:04:18.790 --> 00:04:21.269 us, our definition of what we think about these 00:04:21.269 --> 00:04:25.610 two sort of mindsets actually are and, you know. 00:04:25.610 --> 00:04:29.610 So starting off with like doctors, right. They don't 00:04:29.610 --> 00:04:32.580 treat, rarely, they don't try to treat the symptoms, 00:04:32.580 --> 00:04:34.710 it's about how you deal with them. So they 00:04:34.710 --> 00:04:36.970 go about just figuring out what the problem could 00:04:36.970 --> 00:04:40.120 be, you know, and then probably, you know, a 00:04:40.120 --> 00:04:43.330 lot of tests, make you run through a few 00:04:43.330 --> 00:04:46.250 tests or try and figure out what's what, if 00:04:46.250 --> 00:04:49.460 indeed it is the problem, and then try and, 00:04:49.460 --> 00:04:53.670 based on that, prescribe a treatment and, of course, 00:04:53.670 --> 00:04:55.660 make sure that you're OK at the end of 00:04:55.660 --> 00:04:58.080 it, right. The symptoms are gone. 00:04:58.080 --> 00:05:01.250 P.S.: And we didn't take a look through a medical textbook, so 00:05:01.250 --> 00:05:02.960 we don't know if this is right, but assuming 00:05:02.960 --> 00:05:06.060 it is, though, this is what worked for us, 00:05:06.060 --> 00:05:06.540 so. 00:05:06.540 --> 00:05:10.620 A.K.: And, again, in contrast, a pharmacist's job, 00:05:10.620 --> 00:05:14.360 we think very different, should be more about understanding 00:05:14.360 --> 00:05:18.600 the medicines, the medicines themselves. Probably even figure out 00:05:18.600 --> 00:05:21.000 what the disease is based on the medicine, right. 00:05:21.000 --> 00:05:24.850 But definitely they don't really think about, you know, 00:05:24.850 --> 00:05:26.790 what was the problem originally or what are we 00:05:26.790 --> 00:05:29.420 prescribing the treatment for. And they usually don't do 00:05:29.420 --> 00:05:31.220 it. Hopefully they don't do it. 00:05:31.220 --> 00:05:32.120 P.S.: Yeah. OK. 00:05:32.120 --> 00:05:33.880 So now with this context of what we mean 00:05:33.880 --> 00:05:37.560 by a doctor and a pharmacist and medicines and 00:05:37.560 --> 00:05:40.580 self-medication, all right. Let's get back to our mistake, 00:05:40.580 --> 00:05:44.310 which we want to talk about. Right. So our 00:05:44.310 --> 00:05:45.670 mistake that we want to talk about is a 00:05:45.670 --> 00:05:47.610 way we have dealt with, or rather we used 00:05:47.610 --> 00:05:50.780 to deal with bad symptoms on our code bases 00:05:50.780 --> 00:05:54.630 and on our projects, right. So you, a lot 00:05:54.630 --> 00:05:57.680 of times you see these in your code bases. 00:05:57.680 --> 00:06:01.130 There's the symptom, or there are some issues, right. 00:06:01.130 --> 00:06:04.360 So we obviously had a lot of those, in 00:06:04.360 --> 00:06:07.080 every single project we have worked on, and this 00:06:07.080 --> 00:06:09.310 is about how we dealt with that, right. 00:06:09.310 --> 00:06:12.740 A.K.: Let's start off with one very simple one, or 00:06:12.740 --> 00:06:14.690 at least the one which was the most easily 00:06:14.690 --> 00:06:15.440 expressible. 00:06:15.440 --> 00:06:18.930 P.S.: Yeah, as in, Tejas Dinkar, who has 00:06:18.930 --> 00:06:21.560 already been mentioned several times in different talks, he 00:06:21.560 --> 00:06:24.380 threatened us to throw off the stage if we 00:06:24.380 --> 00:06:28.410 took anything more than twenty-nine minutes, fifty-nine seconds. So 00:06:28.410 --> 00:06:32.390 we had to like really dumbify our, you know, 00:06:32.390 --> 00:06:34.890 anecdotes. But you learn, we have like a quick 00:06:34.890 --> 00:06:36.440 mention of different things which we would love to 00:06:36.440 --> 00:06:40.210 talk about offline, but yeah. So thanks Dejas. 00:06:40.210 --> 00:06:43.270 A.K.: So we'll get started on the first one. So 00:06:43.270 --> 00:06:45.800 this was a project where we had a familiar 00:06:45.800 --> 00:06:49.340 problem of regression bugs. We added new features, and 00:06:49.340 --> 00:06:53.460 that kept breaking things. So this is what we 00:06:53.460 --> 00:06:56.120 designed. We want this down. We want the number 00:06:56.120 --> 00:06:58.650 of bugs down from 10 to 5, you know. 00:06:58.650 --> 00:07:01.250 At that time it was, like, let's set up 00:07:01.250 --> 00:07:03.410 this goal, let's try and achieve this. What did 00:07:03.410 --> 00:07:05.410 we do? Oh, before that. Some facts about the 00:07:05.410 --> 00:07:09.470 project, right. This was not a project we started 00:07:09.470 --> 00:07:11.810 on from scratch, it was a lot of legacy 00:07:11.810 --> 00:07:13.280 code, a lot of code that we did not 00:07:13.280 --> 00:07:17.560 understand, and probably that's why we thought it was 00:07:17.560 --> 00:07:19.280 bad. And the test coverage was- 00:07:19.280 --> 00:07:21.910 P.S.: How many of you have taken over a code base from 00:07:21.910 --> 00:07:24.160 another team and thought the code base was awesome? 00:07:24.160 --> 00:07:29.110 Very small samples, so you realize what we mean 00:07:29.110 --> 00:07:32.320 when we thought it was bad. So. 00:07:32.320 --> 00:07:33.070 A.K.: Sure. 00:07:33.070 --> 00:07:36.200 So test-coverage was low, which was probably one of 00:07:36.200 --> 00:07:39.470 the reasons why people complained about the code base, 00:07:39.470 --> 00:07:41.990 of course. So what's your guess? 00:07:41.990 --> 00:07:46.090 P.S.: OK, so the problem we had was every time we checked 00:07:46.090 --> 00:07:48.640 in something, built a new feature, touch any part 00:07:48.640 --> 00:07:50.330 of the code base, we ended up breaking a 00:07:50.330 --> 00:07:52.280 whole bunch of other things. And we would not 00:07:52.280 --> 00:07:54.300 even know it right away, we would know it 00:07:54.300 --> 00:07:55.240 over a period of time. Right, so this was 00:07:55.240 --> 00:07:57.930 a regression problem. And given the facts, what would 00:07:57.930 --> 00:07:59.000 you probably have done? 00:07:59.000 --> 00:08:00.860 A.K.: I'll try and just 00:08:00.860 --> 00:08:03.910 go back again, and then hopefully forward - we're 00:08:03.910 --> 00:08:04.680 done [00:08:04]??. 00:08:04.680 --> 00:08:07.750 P.S.: Yeah, based on some facts. 00:08:07.750 --> 00:08:13.820 A.K.: Sure, so the low coverage was definitely a problem. 00:08:13.820 --> 00:08:17.040 We thought, I mean, everybody agreed that we need 00:08:17.040 --> 00:08:19.880 to start working on that, and you know, fix 00:08:19.880 --> 00:08:23.340 the coverage. So we went in there, put the 00:08:23.340 --> 00:08:26.150 coverage tool in place, you know. And then we 00:08:26.150 --> 00:08:28.170 decided we will write tests for every bug we 00:08:28.170 --> 00:08:32.610 caught. We got the coverage up, not very surprising, 00:08:32.610 --> 00:08:36.279 I mean, we didn't manage to- you know, improving 00:08:36.279 --> 00:08:37.200 the coverage- 00:08:37.200 --> 00:08:41.039 P.S.: Yeah like, so we spend like the whole time- 00:08:41.039 --> 00:08:42.539 A.K.: over a period of time. 00:08:42.539 --> 00:08:44.880 P.S.: OK, so, like so this was a problem, 00:08:44.880 --> 00:08:46.360 right. When you look at that, when you try 00:08:46.520 --> 00:08:48.380 to abstract the, into, like what our thought process 00:08:48.440 --> 00:08:56.540 was. It was something like this, right. Check. Yeah. 00:08:56.540 --> 00:08:58.620 So, we had a symptom. In this case it 00:08:58.620 --> 00:09:04.140 was low test-coverage, and - Jake - and we 00:09:04.140 --> 00:09:06.750 had, we decided on what metric and tool to 00:09:06.750 --> 00:09:09.700 use. In our case it was a simple line 00:09:09.700 --> 00:09:11.570 coverage, right and, archive?? [00:09:10], this was back in 00:09:11.570 --> 00:09:14.930 the days. And then we started solving that problem 00:09:14.930 --> 00:09:17.550 that we had, and then, you know, hopefully, for 00:09:17.550 --> 00:09:20.670 example, we were TDDing most of the new code 00:09:20.670 --> 00:09:23.970 that we wrote, so coverage was good on that, 00:09:23.970 --> 00:09:28.500 then start writing tests for things, which were, any 00:09:28.500 --> 00:09:29.980 part of the code base which we touched where 00:09:29.980 --> 00:09:31.720 there were no tests, we started adding tests there. 00:09:31.720 --> 00:09:33.900 You know, a bunch of different things. So basically, 00:09:33.900 --> 00:09:36.240 like, the idea was to improve the coverage and 00:09:36.240 --> 00:09:39.620 keep on writing, right. So cool, so. And what 00:09:39.620 --> 00:09:42.710 was the result? We ended up with, of course, 00:09:42.710 --> 00:09:45.390 drastically improving our test coverage, so we were in 00:09:45.390 --> 00:09:48.130 the late '90s for most the part of the 00:09:48.130 --> 00:09:50.630 code base, which was awesome. 00:09:50.630 --> 00:09:53.010 A.K.: A hundred, man, a hundred. 00:09:53.010 --> 00:09:55.890 P.S.: A hundred, yeah. Or, yeah, sure. 00:09:55.890 --> 00:10:00.070 Very good coverage. But things got only marginally better. 00:10:00.070 --> 00:10:02.200 At this point, this was when we realized that 00:10:02.200 --> 00:10:04.260 inspite of putting so much effort into actually improving 00:10:04.260 --> 00:10:07.000 our test coverage, our actual goal was to actual 00:10:07.000 --> 00:10:10.640 reduce the number of regression bugs. So we, we 00:10:10.640 --> 00:10:13.500 were still no better than what we started off, 00:10:13.500 --> 00:10:17.240 about two months back. So the developers were generally 00:10:17.240 --> 00:10:19.580 very happy, now they were doing a lot more 00:10:19.580 --> 00:10:22.360 TDD, they have a lot, they had manag- successfully 00:10:22.360 --> 00:10:24.600 convinced the manag- you know the product manager and 00:10:24.600 --> 00:10:27.590 the stake holders to spend more time on the 00:10:27.590 --> 00:10:29.600 tech deck?? [00:10:29] and things like that. They they 00:10:29.600 --> 00:10:31.800 also were happy. But the project manager was extremely 00:10:31.800 --> 00:10:34.430 frustrated, because in spite of spending so much effort, 00:10:34.430 --> 00:10:37.430 there was no real benefit from any of it, 00:10:37.430 --> 00:10:39.560 right. So it's like one of those very classic 00:10:39.560 --> 00:10:41.730 moments where you know, in Dilbert where, you know, 00:10:41.730 --> 00:10:45.029 the, OK, in Dilbert developers are never happy, but 00:10:45.029 --> 00:10:47.080 at least here we were happy and the project 00:10:47.080 --> 00:10:48.960 manager was sad, right. A.K.: And we weren't happy 00:10:48.960 --> 00:10:49.870 with the project managers as well. 00:10:49.870 --> 00:10:52.510 P.S.: Yeah, and eventually we feel there was something wrong, 00:10:52.510 --> 00:10:56.200 it's not like we take pleasure out of it. So, we 00:10:56.200 --> 00:10:58.630 think this is a very big mistake, where we 00:10:58.630 --> 00:11:01.380 spent almost two months of time without really realizing 00:11:01.380 --> 00:11:03.870 where we were going wrong, right. So this mistake 00:11:03.870 --> 00:11:07.490 and several mistakes across different projects which Dejas won't 00:11:07.490 --> 00:11:11.620 let us go into, ended up making us realize 00:11:11.620 --> 00:11:15.770 something very basic, right. And that's, this is basically 00:11:15.770 --> 00:11:17.000 what, this is what we were going to say. 00:11:17.000 --> 00:11:18.080 A.K.: OK, tell us a little bit. 00:11:18.080 --> 00:11:20.700 P.S.: If we had like a lightning talk, this is what 00:11:20.700 --> 00:11:22.050 we probably the only thing we would have put 00:11:22.050 --> 00:11:26.230 up and left. So never improve a metric. Solving 00:11:26.230 --> 00:11:29.850 a problem should automatically imrpve the metric that you're 00:11:29.850 --> 00:11:34.350 measuring, right. So the focus is on, is never 00:11:34.350 --> 00:11:36.700 on making a metric better. It's always about solving 00:11:36.700 --> 00:11:38.270 the problem. And, the metric- 00:11:38.270 --> 00:11:38.690 A.K.: This is like 00:11:38.690 --> 00:11:40.430 one of those, one of those things that is, 00:11:40.430 --> 00:11:41.620 it's very easily said and- 00:11:41.620 --> 00:11:42.080 P.S.: Yeah, and it- 00:11:42.080 --> 00:11:43.480 A.K.: You have to, at least for us, we 00:11:43.480 --> 00:11:45.070 always fell in that trap of- 00:11:45.070 --> 00:11:45.900 P.S.: Yeah, like 00:11:45.900 --> 00:11:48.480 it almost sounds like 'do the right thing,' but, 00:11:48.480 --> 00:11:51.090 yeah, like, it's very, it fits your common sense 00:11:51.090 --> 00:11:52.670 very well, but then when you're caught up in 00:11:52.670 --> 00:11:55.000 the daily, the day-to-day stuff in what you do 00:11:55.000 --> 00:11:57.029 in a project, it becomes very easy for you 00:11:57.029 --> 00:11:59.430 to miss the point here. So yeah, like this 00:11:59.430 --> 00:12:03.120 is really what, what is essence of what we 00:12:03.120 --> 00:12:05.779 are trying to say, right. 00:12:05.779 --> 00:12:08.190 A.K.: So, so what really happened here. 00:12:08.190 --> 00:12:09.050 Let's go a little bit more 00:12:09.050 --> 00:12:11.940 into what we were trying earlier and what we 00:12:11.940 --> 00:12:14.770 think we should have probably done. Instead of something 00:12:14.770 --> 00:12:18.800 like this, which, which actually ended up attacking the 00:12:18.800 --> 00:12:23.279 symptoms, or you know, targeting the symptom, we want 00:12:23.279 --> 00:12:25.430 to do something like this: There is a symptom, 00:12:25.430 --> 00:12:26.360 so just like always- 00:12:26.360 --> 00:12:27.480 P.S.: This is where the 00:12:27.480 --> 00:12:29.570 whole, like, the pharmacist and the doctor approach, yeah, 00:12:29.570 --> 00:12:32.279 like, it's a very long-shot metaphor, we agree, but- 00:12:32.279 --> 00:12:35.170 A.K.: The doctor thinking, which we hopefully want to 00:12:35.170 --> 00:12:39.800 do, is first is just try and take a 00:12:39.800 --> 00:12:41.339 guess at least at the problem, at least in 00:12:41.339 --> 00:12:42.810 our context, maybe not the doctor's. But in our 00:12:42.810 --> 00:12:46.570 context, take a quess at the problem, right. Think 00:12:46.570 --> 00:12:49.720 what it might be. Then that hopefully will tell 00:12:49.720 --> 00:12:52.230 you what you could do to probably solve the 00:12:52.230 --> 00:12:55.320 problem, solve, you know, that could be the solution 00:12:55.320 --> 00:12:59.230 which hope- will hopefully fix the problem, right. So 00:12:59.230 --> 00:13:01.740 this kind of very similar, we are iterating??[00:13:00] over 00:13:01.740 --> 00:13:05.380 this and hopefully, you know, improving on what we 00:13:05.380 --> 00:13:09.589 thought was the problem. And how do you know, 00:13:09.589 --> 00:13:12.420 then, that you know we are in fact improving 00:13:12.420 --> 00:13:13.870 on the problem? How do we know that is 00:13:13.870 --> 00:13:13.900 the problem? 00:13:13.900 --> 00:13:14.050 P.S.: Like the whole, how do you 00:13:14.050 --> 00:13:14.650 define them, right. So, how do we define them? 00:13:14.650 --> 00:13:17.900 A.K.: And that's where we think, that's where we 00:13:17.900 --> 00:13:21.420 think the metrics come in. Again, not metric, hopefully 00:13:21.420 --> 00:13:24.410 metrics, because that lets us measure the problem. It 00:13:24.410 --> 00:13:27.620 tells us at every point that, you know, you're 00:13:27.620 --> 00:13:30.440 doing better, you know, it's improving. And hopefully you 00:13:30.440 --> 00:13:34.620 also, when it gets done, right. So, yeah. So 00:13:34.620 --> 00:13:36.700 this is, this is probably the approach that we 00:13:36.700 --> 00:13:40.020 would like to try on, try from now on, 00:13:40.020 --> 00:13:43.860 also. And, right, there, you know, the problem may 00:13:43.860 --> 00:13:46.050 not be the, what is the one that we 00:13:46.050 --> 00:13:48.400 started out to fix. Like, like, probably in our 00:13:48.400 --> 00:13:50.610 previous case, you know, it, you should always be 00:13:50.610 --> 00:13:53.270 open to the idea that the problem will, what 00:13:53.270 --> 00:13:54.990 you thought was the problem was never the case, 00:13:54.990 --> 00:13:57.670 and it was not showing up. I mean, you 00:13:57.670 --> 00:14:00.029 were trying to, you were seeing the metrics improve, 00:14:00.029 --> 00:14:02.360 but then the symptom never went away, right. So 00:14:02.360 --> 00:14:04.040 be open to the notion that the problem could 00:14:04.040 --> 00:14:06.520 be different, in which case, the important thing is 00:14:06.520 --> 00:14:09.170 the solution is different and the metrics are different, 00:14:09.170 --> 00:14:10.380 right. So yeah. 00:14:10.380 --> 00:14:12.000 P.S.: Any guesses on what could 00:14:12.000 --> 00:14:15.240 have been the problem on that project? Where the 00:14:15.240 --> 00:14:20.460 regression bugs were written very high? It was duplication, 00:14:20.460 --> 00:14:23.180 as in there was, like, rampant duplication all over 00:14:23.180 --> 00:14:26.270 the place. And we would change something but forget 00:14:26.270 --> 00:14:28.410 to change the same thing in some other place. 00:14:28.410 --> 00:14:30.110 But, and because we didn't know about the code 00:14:30.110 --> 00:14:33.610 base, yeah. We were just blindly adding tests. And 00:14:33.610 --> 00:14:36.710 incrimentally going through different places, where each place where 00:14:36.710 --> 00:14:38.870 we found that there were no tests, we added 00:14:38.870 --> 00:14:40.470 a test, right. But that didn't really help us 00:14:40.470 --> 00:14:42.870 with actually solving the problem of duplication. Yeah. So- 00:14:42.870 --> 00:14:44.730 A.K.: The coverage number is something which is, it 00:14:44.730 --> 00:14:47.300 easily drives you to just keep adding the specs, 00:14:47.300 --> 00:14:50.210 and we will talk about, more about that soon. 00:14:50.210 --> 00:14:53.440 P.S.: Yeah, so, if you really think about it, 00:14:53.440 --> 00:14:55.980 of, basically what, at least, we would love to 00:14:55.980 --> 00:14:59.490 believe that we have stopped doing this, right. So 00:14:59.490 --> 00:15:02.490 Yogi mentioned this in the panel discussion yesterday, block-force 00:15:02.490 --> 00:15:05.779 driven decisions, right. So that was really what we 00:15:05.779 --> 00:15:07.870 were doing, essentially. Like, we were a bunch of 00:15:07.870 --> 00:15:10.899 kids on this project, who'd see a problem at 00:15:10.899 --> 00:15:14.220 the first, the, at the first, trigger we would 00:15:14.220 --> 00:15:17.399 just start crawling the web, start crawling block force, 00:15:17.399 --> 00:15:20.600 see different github projects, find a gem, install it, 00:15:20.600 --> 00:15:23.270 start, like, monitoring it, measuring it, try to improve 00:15:23.270 --> 00:15:25.510 it, you know. We don't really spending too much 00:15:25.510 --> 00:15:28.850 time into figuring out what was really the problem, 00:15:28.850 --> 00:15:35.850 right. And, especially, so in, OK - where does- 00:15:36.960 --> 00:15:40.399 Especially so in Rails projects, where, you know, or 00:15:40.399 --> 00:15:42.589 Ruby projects, where we believe that the number of 00:15:42.589 --> 00:15:46.800 gems which actually bundle best practices is actually very 00:15:46.800 --> 00:15:48.620 high, right. Here it's very easy for us to 00:15:48.620 --> 00:15:50.600 fall into the trap of, OK, just choose a 00:15:50.600 --> 00:15:53.360 gem, start using it, and then two months or 00:15:53.360 --> 00:15:54.690 three months down the way you have no idea 00:15:54.690 --> 00:15:56.589 where you used it in the first place. But 00:15:56.589 --> 00:15:58.580 it's just there in your process, right. Yeah. Like 00:15:58.580 --> 00:16:01.020 at least we used to find ourselves in that 00:16:01.020 --> 00:16:04.600 trap all the time. So yeah. This is basically 00:16:04.600 --> 00:16:08.200 what we stopped doing. OK, at this- 00:16:08.200 --> 00:16:08.820 A.K.: [indecipherable] 00:16:08.820 --> 00:16:10.740 P.S.: So this, this dude is basically Hari, 00:16:10.740 --> 00:16:13.680 he's sitting way over there. Yesterday we were doing 00:16:13.680 --> 00:16:16.600 a write-in??[00:16:14], and the only dude that, after this 00:16:16.600 --> 00:16:18.810 point it's fine, but it's getting monotonous, it's very 00:16:18.810 --> 00:16:21.760 black and white. And you need more images. And, 00:16:21.760 --> 00:16:24.690 like, Jake and I were really, we really think 00:16:24.690 --> 00:16:24.959 that- 00:16:24.959 --> 00:16:26.290 A.K.: That's our image, Hari! 00:16:26.290 --> 00:16:28.720 P.S.: We really think that we don't know how to add images 00:16:28.720 --> 00:16:31.290 to a presentation. And we were like, OK fine 00:16:31.290 --> 00:16:34.709 Hari, we'll just add your picture. And, yeah, so 00:16:34.709 --> 00:16:37.190 thanks for- and as you notice, we are very 00:16:37.190 --> 00:16:38.850 receptive of feedback, so. 00:16:38.850 --> 00:16:41.110 A.K.: He hasn't spoken yet, 00:16:41.110 --> 00:16:42.250 but his is full of interesting ones. 00:16:42.250 --> 00:16:44.959 P.S.: Like it would have been funny if like his was 00:16:44.959 --> 00:16:46.649 the presentation before we got the schedule, but yeah, 00:16:46.649 --> 00:16:51.080 anyway. He'll be talking next. So yeah, when you 00:16:51.080 --> 00:16:53.279 look at test coverage, right, how many of you 00:16:53.279 --> 00:16:59.010 think you understand test coverage very well? Well enough. 00:16:59.010 --> 00:17:03.970 OK, I mean, sure, yeah, like- 00:17:03.970 --> 00:17:07.510 A.K.: This was one thing which at least took us as a 00:17:07.510 --> 00:17:10.419 very obvious metrics thing, which we always get into, 00:17:10.419 --> 00:17:10.609 and- 00:17:10.609 --> 00:17:12.250 P.S.: When we were young and stupid as 00:17:12.250 --> 00:17:14.299 against now old and stupid, right, we were, we 00:17:14.299 --> 00:17:16.500 used to think oh, test coverage, what's that, it's 00:17:16.500 --> 00:17:18.799 just - meh. It's so easy, right. But then 00:17:18.799 --> 00:17:21.839 we realized even something so seemingly obvious had, like, 00:17:21.839 --> 00:17:25.059 so many different shades of details. And once you 00:17:25.059 --> 00:17:27.669 start understanding it and interpreting it in context is 00:17:27.669 --> 00:17:32.029 when you really understand the, the complexity of that, 00:17:32.029 --> 00:17:35.980 of that thing that you're measuring, right. Now think 00:17:35.980 --> 00:17:38.409 of all the things that people at Flipkart measure. 00:17:38.409 --> 00:17:40.820 I don't even know if they have a rational 00:17:40.820 --> 00:17:43.940 behind every one of it. I'm hoping they do, 00:17:43.940 --> 00:17:46.129 but you know like, it becomes very OK, we 00:17:46.129 --> 00:17:48.239 just need to monitor these ten things. Why? Why 00:17:48.239 --> 00:17:51.080 are you doing it? Right. So it should not 00:17:51.080 --> 00:17:52.940 be like a checklist at every project on start. 00:17:52.940 --> 00:17:55.519 You just start using it. So yeah, test coverage 00:17:55.519 --> 00:17:57.779 is definitely one thing that we found where, on 00:17:57.779 --> 00:17:59.059 start of every project we just set up a 00:17:59.059 --> 00:18:01.350 coverage tool. We just wanted to talk about some 00:18:01.350 --> 00:18:04.450 details on what we learned when doing that, so 00:18:04.450 --> 00:18:05.309 yeah. 00:18:05.309 --> 00:18:07.330 A.K.: So first we want to start off 00:18:07.330 --> 00:18:11.850 the obvious one. The measuring. So controller specs versus 00:18:11.850 --> 00:18:15.989 model specs coverage. I- I'm guessing it, does it 00:18:15.989 --> 00:18:20.989 ring any bells? I'm- so, so I'm, think of 00:18:20.989 --> 00:18:23.519 it this way, like, he has a question for 00:18:23.519 --> 00:18:27.320 you, right? You have, let's take a simple case. 00:18:27.320 --> 00:18:29.190 There is a single controller, you have a spec 00:18:29.190 --> 00:18:32.090 around it, there's a corresponding module, you have a 00:18:32.090 --> 00:18:34.279 spec around it, right. And then, you, with these 00:18:34.279 --> 00:18:35.330 two tests you run your coverage, right. 00:18:35.330 --> 00:18:36.649 P.S.: And you get some coverage from- 00:18:36.649 --> 00:18:39.059 A.K.: I'm guessing it's going to be a hundred percent. 00:18:39.059 --> 00:18:43.779 P.S.: Do you see a problem with this? Could there, rather, OK, 00:18:43.779 --> 00:18:49.320 could there be a problem with this? 00:18:49.320 --> 00:18:53.669 A.K.: What if you just removed the model spec? 00:18:53.669 --> 00:18:56.960 P.S.: What will the coverage for model be? Is there a 00:18:56.960 --> 00:19:01.570 chance that model's coverage is not zero? 00:19:01.570 --> 00:19:04.570 A.K.: Your controller spec is still gonna be loading the model. 00:19:04.570 --> 00:19:05.049 So your coverage- 00:19:05.049 --> 00:19:05.519 P.S.: Depends on- 00:19:05.519 --> 00:19:06.309 A.K.: -is still in question. 00:19:06.309 --> 00:19:08.139 P.S.: The answer is it depends, right, 00:19:08.139 --> 00:19:09.950 like, really depends on how you are testing your 00:19:09.950 --> 00:19:11.730 controller. But most of all things, what we have 00:19:11.730 --> 00:19:14.409 seen is, not every model is mocked out in 00:19:14.409 --> 00:19:17.169 the controller. Well, it's a totally different debate, whether 00:19:17.169 --> 00:19:19.980 should you mock your models or not, but if 00:19:19.980 --> 00:19:22.629 you are not modelin- or, mocking, your models are 00:19:22.629 --> 00:19:25.330 being loaded in your controller. So the controller spec, 00:19:25.330 --> 00:19:27.789 when it is tested for, like when the coverage 00:19:27.789 --> 00:19:31.289 is reported your models are being reported as well. 00:19:31.289 --> 00:19:33.759 A.K.: While, while the, or the point of, the 00:19:33.759 --> 00:19:35.600 point we are trying to make is, here, is 00:19:35.600 --> 00:19:37.919 not whether you should, how you should test your 00:19:37.919 --> 00:19:41.570 model specs and controller specs. What we do implore 00:19:41.570 --> 00:19:44.220 you to do is make sure that when you're 00:19:44.220 --> 00:19:45.989 doing, when you're looking at your coverage, you do 00:19:45.989 --> 00:19:48.350 have in mind your testing strategy, which is, am 00:19:48.350 --> 00:19:50.549 I actually mocking the model out or is it 00:19:50.549 --> 00:19:52.980 also getting wrote as a part of my controller 00:19:52.980 --> 00:19:54.879 spec? Is my controller spec also hitting the models, 00:19:54.879 --> 00:19:57.169 right? Think about these things when, when you're looking 00:19:57.169 --> 00:20:00.059 at these numbers, right. Or something that worked for 00:20:00.059 --> 00:20:02.179 us which we tried to do was we started 00:20:02.179 --> 00:20:05.730 monitoring the model specs coverage independently, and then started 00:20:05.730 --> 00:20:12.539 looking at the controller specs in light of the, 00:20:12.539 --> 00:20:14.470 in light of the model spec coverage. We wanted 00:20:14.470 --> 00:20:16.820 the model spec coverage to be high, because at 00:20:16.820 --> 00:20:19.649 least we wanted all, hopefully all our business logic 00:20:19.649 --> 00:20:22.590 was in the model specs, and you know, that's 00:20:22.590 --> 00:20:28.960 what we were keen on. Yes. Yeah, and then 00:20:28.960 --> 00:20:31.590 the next one, the line coverage itself, I think 00:20:31.590 --> 00:20:33.320 most commonly when we talk about coverage we just 00:20:33.320 --> 00:20:35.419 talk about line coverage. But then there is a 00:20:35.419 --> 00:20:38.080 bunch of other things as well, branch coverage, and 00:20:38.080 --> 00:20:39.830 then unique path coverage. 00:20:39.830 --> 00:20:40.710 P.S.: How many of you 00:20:40.710 --> 00:20:42.239 pay attention to branch coverages? 00:20:42.239 --> 00:20:44.730 A.K.: Or even monitor it? 00:20:44.730 --> 00:20:46.850 P.S.: How many of you don't think it's 00:20:46.850 --> 00:20:52.690 important? How many of you have no opinions? Cool. 00:20:52.690 --> 00:20:58.190 Yeah. I mean, sure. We have it on projects 00:20:58.190 --> 00:21:00.690 where it's been important, it's not been important, it's 00:21:00.690 --> 00:21:02.739 fine. But it's just that, you just need to 00:21:02.739 --> 00:21:05.830 know that it exists, and you need to train 00:21:05.830 --> 00:21:07.570 your data, right, so. 00:21:07.570 --> 00:21:08.590 A.K.: Just, hopefully it should 00:21:08.590 --> 00:21:11.809 not be a single metric. Something usually seems wrong 00:21:11.809 --> 00:21:13.019 if it is just gonna be about that one 00:21:13.019 --> 00:21:19.080 metric. Next one. So reporting. Yeah, this one is 00:21:19.080 --> 00:21:21.389 a bit tricky. What I usually don't like about 00:21:21.389 --> 00:21:24.480 coverage tools and these tools in general is they 00:21:24.480 --> 00:21:26.929 sometimes miss out this aspect of it. And they, 00:21:26.929 --> 00:21:28.489 in an attempt to try and be nice to 00:21:28.489 --> 00:21:30.820 you when you are very simple answered is try 00:21:30.820 --> 00:21:32.809 and give you a number which inherently makes it 00:21:32.809 --> 00:21:36.070 good or bad. There's nothing in between, and then 00:21:36.070 --> 00:21:38.960 the focus is lost. Like you either start liking 00:21:38.960 --> 00:21:41.340 it or you don't like it. You don't really 00:21:41.340 --> 00:21:46.149 think about what is there in between, right. Yeah. 00:21:46.149 --> 00:21:47.950 So the focus on, focus on some of the 00:21:47.950 --> 00:21:49.499 aspects of where is the coverage, you know, what 00:21:49.499 --> 00:21:51.909 does it mean, right. One thing that worked for 00:21:51.909 --> 00:21:55.109 us was code climate in the region projects. I 00:21:55.109 --> 00:21:56.989 really like it because they put a lot of 00:21:56.989 --> 00:21:59.700 focus into the presentation aspect of it. Not just 00:21:59.700 --> 00:22:03.179 the collection aspect of it. It really, you know, 00:22:03.179 --> 00:22:06.929 takes you down to the, to the code, which 00:22:06.929 --> 00:22:09.649 is missing the specs. Of course, they do also 00:22:09.649 --> 00:22:11.940 other metrics like code quality, which I really like, 00:22:11.940 --> 00:22:14.489 by the way. They have some notification things like 00:22:14.489 --> 00:22:17.109 that, like, on, like Lexus a lot, whenever this 00:22:17.109 --> 00:22:19.169 climate goes in, poof the coverage goes down or 00:22:19.169 --> 00:22:21.019 something like that. [00:22:19] It doesn't break the builder 00:22:21.019 --> 00:22:23.600 part, it doesn't break the build, but it lets 00:22:23.600 --> 00:22:25.080 you know, and then you can deal with it 00:22:25.080 --> 00:22:27.109 if you think it is important to deal with. 00:22:27.109 --> 00:22:30.389 P.S.: OK, speaking of breaking build, how many of 00:22:30.389 --> 00:22:35.859 you know what racheting, in builds? OK. So the 00:22:35.859 --> 00:22:37.999 idea of racheting is basically you will never leave 00:22:37.999 --> 00:22:41.100 your code base worse than what it already was, 00:22:41.100 --> 00:22:45.059 right. So every commit basically makes sure, even if 00:22:45.059 --> 00:22:46.759 it doesn't do any good, it doesn't do any 00:22:46.759 --> 00:22:48.779 bad to your code base. So for example, if 00:22:48.779 --> 00:22:51.859 you're, your current code coverage is at 70%, and 00:22:51.859 --> 00:22:53.840 if this check-in makes it 69%, it will break 00:22:53.840 --> 00:22:56.309 the build. Even though there's nothing functionally wrong with 00:22:56.309 --> 00:23:00.169 it, you know, it's bad, right. We really think 00:23:00.169 --> 00:23:02.600 it's a double-edged sword. I will, this is one 00:23:02.600 --> 00:23:05.249 of those things which I have a, in theory 00:23:05.249 --> 00:23:08.629 sounds very, very good, and direct, but in practice, 00:23:08.629 --> 00:23:10.950 what it typically ends up doing is people end 00:23:10.950 --> 00:23:14.539 up fretting the actual metric, and never about what 00:23:14.539 --> 00:23:17.570 the problem is, right. Becau- this exactly does what 00:23:17.570 --> 00:23:20.450 we said in the previous slide, which is coverage 00:23:20.450 --> 00:23:24.029 is never red or green, right. Sometimes you are 00:23:24.029 --> 00:23:26.359 OK with taking this hit because you want to 00:23:26.359 --> 00:23:28.859 do something. I mean there are all, there are 00:23:28.859 --> 00:23:33.720 so many reasons why which, in practic- in reality 00:23:33.720 --> 00:23:35.850 you may have to do some bad things, and 00:23:35.850 --> 00:23:37.509 eventually have to pay for it, but it's OK, 00:23:37.509 --> 00:23:40.690 it's a conscious decision, right. But racheting invariably stops 00:23:40.690 --> 00:23:44.759 that. It makes it very, you know, black and 00:23:44.759 --> 00:23:45.470 white, right. 00:23:45.470 --> 00:23:47.309 A.K.: Difficult for you to proceed at 00:23:47.309 --> 00:23:47.869 that very moment. 00:23:47.869 --> 00:23:49.369 P.S.: Yeah and it has a 00:23:49.369 --> 00:23:53.379 more behavioral impact on the team, which is your, 00:23:53.379 --> 00:23:56.850 your team members start hating either the racheting, or 00:23:56.850 --> 00:24:00.259 they start hating the metric, or you know, we 00:24:00.259 --> 00:24:03.799 had this one person who did not commit for 00:24:03.799 --> 00:24:06.129 four days because they thought they did not have 00:24:06.129 --> 00:24:09.299 enough test coverage. And, like, when we breached the 00:24:09.299 --> 00:24:12.350 whole, OK, frequent check-ins, call check-in, and this person 00:24:12.350 --> 00:24:14.389 was actually scared because they were about, they would 00:24:14.389 --> 00:24:17.600 break the build, is actually a, very, very bad 00:24:17.600 --> 00:24:20.729 signal, of, a bad sign from your team, right. 00:24:20.729 --> 00:24:22.690 Now well, you can always argue, OK, this person 00:24:22.690 --> 00:24:25.139 totally missed the point, we can say a bunch 00:24:25.139 --> 00:24:27.499 of things, right, but this is the reality. So 00:24:27.499 --> 00:24:29.739 we think that it's a good idea, so that 00:24:29.739 --> 00:24:32.039 you might want to do it at certain points 00:24:32.039 --> 00:24:35.229 in time. But yeah, be very, very careful about 00:24:35.229 --> 00:24:37.729 the freakanomic implication that it has on your team, 00:24:37.729 --> 00:24:38.080 right, always keep that in mind. 00:24:38.080 --> 00:24:40.419 A.K.: And then 00:24:40.419 --> 00:24:43.269 the very other popular thing, which is, you just 00:24:43.269 --> 00:24:46.109 write that test to bump up the coverage, if 00:24:46.109 --> 00:24:46.679 you're not just- 00:24:46.679 --> 00:24:47.309 P.S.: Yeah, so there was, there 00:24:47.309 --> 00:24:49.769 was just one more philosophy of full-on coverage?? [00:24:50], 00:24:49.769 --> 00:24:52.429 which is saying, OK, you just checked-in code, but 00:24:52.429 --> 00:24:54.249 you don't have a test for this, but that's 00:24:54.249 --> 00:24:56.669 OK. There's this other class which does not have 00:24:56.669 --> 00:24:58.669 a test for it, which is easy, so let's 00:24:58.669 --> 00:25:01.479 start this there and keep my coverage maintained right 00:25:01.479 --> 00:25:04.049 there. There's so many, like, weird things people end 00:25:04.049 --> 00:25:06.389 up doing, just because they are now worried about 00:25:06.389 --> 00:25:09.119 coverage and not really worried about what it means 00:25:09.119 --> 00:25:11.470 to- you know, what it means with what they 00:25:11.470 --> 00:25:12.419 are doing, right. So it's- 00:25:12.419 --> 00:25:14.759 is done with the best of intentions, but then 00:25:14.759 --> 00:25:16.749 you get really, really bad, cause, like- 00:25:16.749 --> 00:25:17.320 P.S.: Yeah. 00:25:17.320 --> 00:25:19.879 So, OK, how, how did we go, I mean, 00:25:19.879 --> 00:25:22.440 how do we now improve, like, so we still 00:25:22.440 --> 00:25:25.639 take on a lot of projects, customer, we are, 00:25:25.639 --> 00:25:27.600 we are mainly now in consulting, so we do 00:25:27.600 --> 00:25:29.889 end up taking worker code bases, right. So what 00:25:29.889 --> 00:25:32.389 do we do to improve coverage if we have 00:25:32.389 --> 00:25:34.690 a bad one? One thing we realized is adding 00:25:34.690 --> 00:25:37.950 unit tests to classes, existing classes, could be a 00:25:37.950 --> 00:25:41.200 very dangerous thing, right. What that essentially means, is 00:25:41.200 --> 00:25:43.369 you have, you know, you are cementing the current 00:25:43.369 --> 00:25:46.379 design. Now I won't say, like prejudiously, it might 00:25:46.379 --> 00:25:48.639 be bad to say not a good design, right. 00:25:48.639 --> 00:25:50.989 But you are cementing it, right. If ever you 00:25:50.989 --> 00:25:54.470 want to cement something, cement features. Cement functionality, right. 00:25:54.470 --> 00:25:56.529 Which means you might want to write like a 00:25:56.529 --> 00:26:00.059 much higher level test of, and, you know, ensure 00:26:00.059 --> 00:26:01.799 that the functionality is the same, so that you 00:26:01.799 --> 00:26:04.239 can go and refactor later. On this one project, 00:26:04.239 --> 00:26:07.009 which Yogi, me, and Jake worked together on in 00:26:07.009 --> 00:26:11.470 ThoughtWorks, it worked beautifully for us, where we were 00:26:11.470 --> 00:26:15.879 strangling ?? to Hibernate [00:26:14]. Which meant that all 00:26:15.879 --> 00:26:18.840 that entire unit tests and database level tests were 00:26:18.840 --> 00:26:21.960 completely invalidated, they were useless, right. And because of 00:26:21.960 --> 00:26:24.309 the way, and because we wanted to turn on 00:26:24.309 --> 00:26:26.470 to caches?? [00:26:24], transactions changed. Like that meant the 00:26:26.470 --> 00:26:30.100 whole app was rendered useless from a testing point 00:26:30.100 --> 00:26:31.359 of view, right, from a safety net point of 00:26:31.359 --> 00:26:34.979 view. But what came to our help was controller 00:26:34.979 --> 00:26:38.289 and beyond level tests. So our biggest, we had 00:26:38.289 --> 00:26:40.929 such good coverage there, that we went in and 00:26:40.929 --> 00:26:42.989 we just modified a whole bunch of things, and 00:26:42.989 --> 00:26:45.399 we like started deleting tests, you know. You get 00:26:45.399 --> 00:26:48.289 a lot more flexibility and freedom inside your code 00:26:48.289 --> 00:26:49.840 base when you know that you're not breaking any 00:26:49.840 --> 00:26:53.210 functionality. So yeah like it's a really, really, like, 00:26:53.210 --> 00:26:55.570 good thing, so definitely think about it when you're 00:26:55.570 --> 00:26:57.229 inheriting a legacy codebase. 00:26:57.229 --> 00:26:58.669 A.K.: Unit tests, unit tests 00:26:58.669 --> 00:27:01.279 are great, but do keep in mind that they 00:27:01.279 --> 00:27:03.029 might also come in the way of- 00:27:03.029 --> 00:27:03.619 P.S.: Changes. 00:27:03.619 --> 00:27:04.489 A.K.: Refactoring, and changes. 00:27:04.489 --> 00:27:04.909 P.S.: Yeah. 00:27:04.909 --> 00:27:05.899 A.K.: Big, big refactoring- 00:27:05.899 --> 00:27:07.739 P.S.: So, OK. What - the whole measurement 00:27:07.739 --> 00:27:11.399 reporting, racheting, improvement, all of this is basically saying, 00:27:11.399 --> 00:27:13.279 always keep the problem you're solving in mind, right. 00:27:13.279 --> 00:27:15.739 Rachet don't rachet - how do you decide? Well, 00:27:15.739 --> 00:27:17.950 is it helping you to achieve your goal, or 00:27:17.950 --> 00:27:20.460 the problem you have at hand? Sure, you know, 00:27:20.460 --> 00:27:26.499 so do it. Otherwise don't do it, right. It's- 00:27:26.499 --> 00:27:27.249 Yeah. 00:27:27.249 --> 00:27:28.659 A.K.: I think- 00:27:28.659 --> 00:27:29.999 P.S.: So, the second- 00:27:29.999 --> 00:27:30.669 A.K.: Let's- 00:27:30.669 --> 00:27:33.350 P.S.: -anecdote- I'll just be, quickly talk about- 00:27:33.350 --> 00:27:35.359 A.K.: We have five minutes, so- 00:27:35.359 --> 00:27:36.359 P.S.: Yeah, like, 00:27:36.359 --> 00:27:37.369 the second anecdote was basically, OK, we had this 00:27:37.369 --> 00:27:42.489 server which was under a very heavy load. And, 00:27:42.489 --> 00:27:48.629 like thousands of requests, a minute, and only about 00:27:48.629 --> 00:27:52.320 5% of those requests, in very seemingly arbitrary periods 00:27:52.320 --> 00:27:56.340 of times, and arbirtrary controllers and actions, would pause, 00:27:56.340 --> 00:27:59.409 and it would take a very long time, right. 00:27:59.409 --> 00:28:01.899 So the- this problem was way more technical. It 00:28:01.899 --> 00:28:04.169 had nothing to do with the behavior or, you 00:28:04.169 --> 00:28:09.440 know, like, or practices part of, or side of 00:28:09.440 --> 00:28:11.460 things. It was pure technical problem. And goal was 00:28:11.460 --> 00:28:13.059 for us to find the root cause of this 00:28:13.059 --> 00:28:17.200 unpredictable behavior and fix it, right. And, yeah, like 00:28:17.200 --> 00:28:20.340 we were, like we can definitely talk about how 00:28:20.340 --> 00:28:22.499 we went into, like a lot of different solving, 00:28:22.499 --> 00:28:24.549 a lot of different symptoms, at one point even 00:28:24.549 --> 00:28:28.159 suspecting JRuby. You know, like, so it's very, like, 00:28:28.159 --> 00:28:30.580 it becomes very hard for you to figure out, 00:28:30.580 --> 00:28:32.470 OK, what is a problem, what is a symptom, 00:28:32.470 --> 00:28:35.159 and like, go very methodical about it, right. So 00:28:35.159 --> 00:28:38.009 that's what this, this problem was going to be 00:28:38.009 --> 00:28:38.999 about. But let's take this offline. At this point 00:28:38.999 --> 00:28:39.029 we can- 00:28:39.029 --> 00:28:39.139 A.K.: I mean, before that, yeah, let's 00:28:39.139 --> 00:28:43.159 get out some questions. 00:28:43.159 --> 00:28:45.839 P.S.: Yeah, let's take some 00:28:45.840 --> 00:28:47.720 questions if you have any. 00:28:47.720 --> 00:28:48.920 A.K.: Cool. 00:28:48.920 --> 00:28:50.520 P.S.: Cool, thanks. 00:28:50.520 --> 00:28:51.720 A.K.: Thanks. 00:28:51.720 --> 00:28:53.180 P.S.: Thanks guys.