Home Subscribe

2. Makov Process

The future evolution of the process depends only on the instant observation of the system and not on the past.

2.1. A 2-state Markov chain

Idea

Example 1

Assume you have 2 shirts, white and blue. Their state space is \(S = {W,B}\).

- The probability of you wearing a white shirt and continue wearing the white shirt is 0.7.
- The probability of changing the white shirt to a blue shirt is 0.3.
- Once you’re wearing a blue shirt, the probability of you continue wearing the blue shirt is 0.4.
- The probability of changing the blue shirt to the white shirt is 0.6.

Solution:
\[\text{2 Shirts} = \begin{cases} P(W|W) &= 0.7\\ P(B|W) &= 0.3\\ P(W|B) &= 0.6\\ P(B|B) &= 0.4 \end{cases}\]
  • Suppose you repeat this for several days. Let’s say today is Monday and you’ve already decided that you’ll wear a white shirt. \(S = {1,0}\)

  • What is the probability of you wearing a white or blue shirt on Tuesday through Friday?

(i) At the end of Monday: Mon \(= S \times \text{trans_matrix}\)
(ii) At the end of Tuesday: Tue \(= Mon \times \text{trans_matrix}\)
(iii) At the end of Wednesday: Wed \(= Tue \times \text{trans_matrix}\)

1nodes = ['White', 'Blue']
2trans_matrix = np.array([[0.7,0.3],[0.6,0.4]])
3pd.DataFrame(trans_matrix, columns=nodes, index=nodes)
1S = np.array([[1,0]])
2Mon = np.matmul(S,trans_matrix)
3Mon
1Tue = np.matmul(Mon,trans_matrix)
2Tue
  • If you are not decided, you can set \(S = [0.5, 0.5\)] or introduce randomness.

2.2. A 3-state Markov chain

Idea

Example 2

There are 3 states:
- Home
- Bar
- Back Home

- You only have one bar you go to if you want to go out.
- From state Home you can only go out. The first assumption is that you have to go out to be in a bar.
- From the Bar, you can stay in the bar or go back home.
- For each time step, you have a 50% probability of going back home and a 50% probability of staying.
- When you are Back Home, the only thing that you could do is stay home. It means you won’t go out again.

Solution:
 1import numpy as np
 2import pandas as pd
 3
 4nodes = ['Home', 'Bar', 'Back Home']
 5trans_matrix = np.array([[0,1,0],[0,0.5,0.5],[0,0,1]])
 6pd.DataFrame(trans_matrix, columns=nodes, index=nodes)
 7
 8# You're in the bar. Where can you go next?
 9# All nodes: np.arange(0,len(nodes),1)
10# Bar transitions: trans_matrix[1]
11print(np.random.choice(np.arange(0,len(nodes),1), p=trans_matrix[1]))


Add Comment

* Required information
1000
Drag & drop images (max 3)
Enter the word shark backwards.

Comments

No comments yet. Be the first!