
[Thrun] And again the solution follows directly from the state diagram over here.

In the beginning we do know we're in state A

and the chance of remaining in A is 0.5.

This is the 0.5 over here. We can just read this off.

For the next state we find ourselves to be with 0.5 chance to be in A

and 0.5 chance to be in B.

If we're in B, we transition with certainty to A.

That's because of the 0.5.

But if we're in A, we stay in A with a 0.5 chance. So you put this together.

0.5 probability being in A times 0.5 probability of remaining in A

plus 0.5 probability to be in B times 1 probability to transition to A.

That gives us 0.75.

Following the same logic but now we're in A with 0.75 times a 0.5 probability

of staying in A plus 0.25 in B, which is 1 minus 0.75,

and the transition's uncertainty back to A as 1, we get 0.625.

So now you should be able to take a Markov chain and compute by hand

or write a piece of software the probabilities of future states.

You will be able to predict something. That's really exciting.