Erik L - 1 year ago 91
R Question

# Markov chain in R not leaving initial state

My Markov chain simulation will not leave the initial state 1. The 4x4 transition matrix has absorption states 0 and 3.

The same code is working for a 3x3 transition matrix without absorption states. What is wrong?

A plot from my code: Plot of Markov chain simulation. And hopefully a working example:

``````# Building Markov transition matrix)
Pc <- matrix(c(1, 0.05, 0, 0, 0, 0.85, 0.05, 0, 0, 0.1, 0.65, 0, 0, 0, 0.3, 1),
nrow = 4, ncol = 4)
rownames(Pc) <- c(0,1,2,3)
colnames(Pc) <- c(0,1,2,3)

#Simulating Markov chain
markovSimulation <- function(matrix, length, initialState) {
chain <- array(0,c(length,1))
chain[1] <- initialState
for (j in 2:length){
chain[j] <- sample(1:4, size = 1, prob=matrix[chain[length(chain)] + 1, ])
}#for loop
return(chain)
}#markovSimulation

# Calling simulator and plotting result
simulatedChain <- markovSimulation(Pc, 10, 1)
plot(simulatedChain)
``````

welcome to SO! Good effort, but I see you got two things wrong:

``````chain[j] <- sample(1:4, size = 1, prob=matrix[chain[length(chain)] + 1, ])
``````

you need to sample from `c(0,1,2,3)` as you index your states from 0. and the probabilities need to depend on the previous state, so `prob=matrix[chain[j-1]+1]`.

Then I'm getting reasonable results like this:

``````    > markovSimulation(Pc,10,1)
[,1]
[1,]    1
[2,]    1
[3,]    2
[4,]    3
[5,]    3
[6,]    3
[7,]    3
[8,]    3
[9,]    3
[10,]    3
``````

+1 for nice first post, good reproducible example!

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download