Markov decision processes
Not met yet
A Markov decision process describes a problem one step at a time. States: where the agent is. Actions: what it can do there. Transitions: for each state and action, every next state that can follow and its chance, and the chances add up to 1. Rewards: what each step is worth. Markov means the next state depends only on this state and this action, not on the history. A policy says what to do in every state, as a dictionary from state to action, because the agent can't know in advance where chance will put it.
Try it
main.pyOutput
Press Run and what your program prints shows up here.
Easy to get wrong
- Thinks a policy is a fixed list of moves from the start, rather than an action for every state the agent might find itself in.
- Thinks an action always leads to the state it aims at, forgetting the chance of slipping somewhere else.
- Thinks picking one of the possible outcomes at random is the same as picking each with its own chance.
- Thinks bumping into an edge or a rock ends the episode or is free, when the agent stays put and the step still costs its reward.