What is markov theory?

Answer 1

I will rephase your question as "What is a finite Markov chain processes?" If this is not your question, please resubmit it. As I researched this question, I found an interesting information on the Markov's, as his brother and brother's son contributed to mathematics. See: http://en.wikipedia.org/wiki/Andrey_Markov How much of Markov chain theory actually orginated with Andrey or future generations of Markov's and others is a good question. Commonly, mathematicians name their theories to other famous mathematicians, for example, Bayes, Gauss & Bernoulli. A Markov chain is a hypothetical process that has finite and identifiable states. The system changes in discrete steps, and only the current state will have an affect of the future state. I will use a concrete example of this. I'm hungry and standing in line at McDonalds. Suppose there are a finite number of choices, say 1- 12 (yes I know this is not true, but just suppose it, ok). Now suppose that my choice is not completely random, but is affected by what the person ahead of me is choosing. If the person ahead of me chooses a number 1, I will have have a higher chance of choosing a number 1 (say for example 50%) and a lower chance of choosing all other options. I could run a simulation model where thousands of customers are served and find out what are the probabilities of number 1 to 13 being served. The state of the system is determined by the selections of the customer defined by an unchanging set of rules governing the probability. In a more general sense, we could have a fix set of rules (conditional probabilities), that state given what the last customer ordered, will determine the probabilities of the next customer. This is called a transition matrix of probabilities. You can find more examples at: http://en.wikipedia.org/wiki/Examples_of_Markov_chains which cites the board game of monopoly as an example of Markov chain, because given the square that you are currently on, will determine the probabilities of the "future state" or the next square on which you will land.