Is it time sampled and not continuous?
Is the next state random determined by weighted values of various inputs?
Is it a control system with perceived credits or reduction of difference moving towards a goal by the new decision and unbiased by the past?