Computationally, when we solve for the stationary probabilities for a countable-state Markov chain (or a linear system of infinitely many equations), the transition probability matrix of the Markov chain (or the coefficient matrix) has to be truncated, in some way, into a finite one. To make the truncated matrix stochastic (MC) again, various augmentation methods might be valid such that the stationary probability distribution for the augmented Markov chain (solution to the augmented finite matrix) approaches that for the countable Markov chain as the truncation size gets large. In this talk, we introduce the censored Markov chain, one of the truncation method, and show that the censored (watched) Markov chain provides the best approximation in the sense that for a given truncation size the sum of errors is the minimum and show, by examples, that the method of augmenting the last column only is not always the best.

This talk is based on joint work with Dan Liu