Erik L - 8 months ago 50

R Question

My Markov chain simulation will not leave the initial state 1. The 4x4 transition matrix has absorption states 0 and 3.

The same code is working for a 3x3 transition matrix without absorption states. What is wrong?

A plot from my code: Plot of Markov chain simulation. And hopefully a working example:

`# Building Markov transition matrix)`

Pc <- matrix(c(1, 0.05, 0, 0, 0, 0.85, 0.05, 0, 0, 0.1, 0.65, 0, 0, 0, 0.3, 1),

nrow = 4, ncol = 4)

rownames(Pc) <- c(0,1,2,3)

colnames(Pc) <- c(0,1,2,3)

#Simulating Markov chain

markovSimulation <- function(matrix, length, initialState) {

chain <- array(0,c(length,1))

chain[1] <- initialState

for (j in 2:length){

chain[j] <- sample(1:4, size = 1, prob=matrix[chain[length(chain)] + 1, ])

}#for loop

return(chain)

}#markovSimulation

# Calling simulator and plotting result

simulatedChain <- markovSimulation(Pc, 10, 1)

plot(simulatedChain)

Answer

welcome to SO! Good effort, but I see you got two things wrong:

```
chain[j] <- sample(1:4, size = 1, prob=matrix[chain[length(chain)] + 1, ])
```

you need to sample from `c(0,1,2,3)`

as you index your states from 0. and the probabilities need to depend on the previous state, so `prob=matrix[chain[j-1]+1]`

.

Then I'm getting reasonable results like this:

```
> markovSimulation(Pc,10,1)
[,1]
[1,] 1
[2,] 1
[3,] 2
[4,] 3
[5,] 3
[6,] 3
[7,] 3
[8,] 3
[9,] 3
[10,] 3
```

+1 for nice first post, good reproducible example!

Source (Stackoverflow)