So we seen a few examples measuring the time of execution of toy programs, of a loop that does nothing. What we care about though is the time of execution of our index encode, so what I'm going to do next is write a program to be able to test the time of execution of indexing code. To get a good test we need to make a big index, we need to fill up an index with lots and lots of words, and we can do that by hand; that would take a lot of time and effort. So what I'm going to do instead, is to find a procedure that makes a big index. So what it does is take in, a size, and then it's going to, fill up an index with that number of words. To fill up the index with that number of keywords, we need to generate different words. So what I've done is created a variable called letters, that is initially is all A's. And as we go through the loop, we go through the loop size number of times. We keep making a new string. We add that string to the next, I'll explain what make_string does later. We add that string to the index, then we change the letters. And we're going to keep increasing the letter once we get to z, we wrap around. For now it's not too important to understand everything in this code, but I do want to walk through the code a little bit. So what this loop is doing is going through all the positions and the letters are A and we filled this up with eight different elements and we're going to go through those elements starting from the last one going backwards, so this range loop goes from the length minus 1 to 0, stepping by negative 1. We are going to check each letter. If the letter is less than z, that means we can increase and we are going to increase it using this code here and I'll talk soon about the code that turns letters into numbers, but what this expression does is get the next letter. So if the letter was an a the result after this will be a b, and we're going to replace the letter at that position in the letters list, with the next letter. If the letter is a z, we don't want to go beyond the alphabet, so instead, we're going to set that letter to a, and we're going to go back through the loop to find the next letter. Once we found one less than z, we break, we only need to change one letter. What the make_string procedure does, that we call here. Is just turn that array into one string. So it's going through the elements of P which is this list of letters that keeps changing. And concatenating all those letters into one string. So the whole point of this is to allow us to easily make big indexes so we can run tests on different size indexes. So let's try this in the Python shell. First I'll show you what the result is when we use make big index. We'll start with a fairly small one, so I'm passing three is the size. What make big index gives us is an index with three keywords: aaaaaaaa, aaaaaaab, aaaaaaac and for each of them there's one url which is the name fake. If we passing in a bigger value, this will have an index with a hundred keywords, so we're going to passing a hundred is the size, we get this big index, and you can see it's starting to change the second from the last letter. To make sure that each word is a different word than the next. So, what we want to do now is look at how the time of executions take for different size indexes. So, let's make a really big index. So what we're going to do, we'll make an index of size 10,000. And remember, our concern is the time for lookups. It's the operation that's going to happen most frequently. So we're not timing the time to make the big index. Let's see what the time is to do a lookup, and so I'll time the execution of looking up in index 10,000 the keyword, and the word makes a difference. So, first let's try looking up the word judacity. Which, sadly, is not in our index. We'd need a much bigger index to get up to udacity. And the time of execution is shown here. So, it's 0.0008 seconds. So, still getting close to a millisecond, but still quite fast. Let's make a bigger index. This time we have a 100,000 keywords. going to take a little longer to make it, but we're not worrying about the time to make it now. What we care is the time to do a look up. So that took a long time. Let's see how, many entries there are, so we can look at the very last element in our index. And we can see got to aaaafryd, which I don't know to pronounce. Another way to see that, which we didn't talk about yet, we can actually index from the back using negative numbers, so if you use negative 1 as the index, that will give us the last entry in the list. So, now we'll try doing a timed execution. We're going to look up in the 10,000 size index. And we'll see the time's pretty similar to what it was before, that time might vary a little bit, let's try it once more, and again, it's just under a millisecond. So now, we'll try, instead of the 10,000 index, looking up in the 100,000 length index, the same look-up, and we see that the time is now 10 times that, so it's now about 8.6 milliseconds whereas before it was 0.9 milliseconds. And let's, for consistency, try that again. We'll note that these timings vary a little bit, each time we do it. And there's lots of reasons why the timing varies. We're running lots of other things on the computer at the same time. So it's not the case that we have total control over the processor and are running exactly the same thing every time. Because all of the other programs might be doing other things. The other reason the time can vary is where things are in memory. Sometime it's very quick to look up a value in memory, sometimes it takes longer. And we're not going to talk about the details of that. What matters is that, that the time's roughly the same, each time we execute it. And it really depends on the size of the input, in this case it's the size of the input table. So when we increase the size of the table to have 100,000 entires, it's about 10 times as slow as when we had 10,000 entries. So now let's have a few quizzes to see if you can guess how these timings work.