- given a state s and an action a, the resultant state is influenced by s, and a, as well as a random environment influence.

- think of MDPs as graphs where states are nodes with actions to chance nodes. 

- because randomness exists, instead of "paths", we have "policies" 

MDPs involve environment where outcome is partly random, and partly influenced by user's decisions.