Paul's Python CoursePPC

Policy iteration

Not met yet

Policy iteration finds the best policy by taking turns. Evaluate the current policy to get its values. Improve it: in every state, take the action that looks best one step ahead on those values (the greedy policy). Then evaluate the new policy, and so on, until improving gives back the same policy, which is then the best. Each improvement is a real one, so it needs few rounds, but every round contains a whole policy evaluation.

Try it

main.py

Output

Press Run and what your program prints shows up here.

Easy to get wrong

  • Thinks one round of policy improvement fixes the whole policy at once, when each round only improves on the values of the last policy.
  • Thinks policy iteration runs a fixed number of rounds, rather than stopping when improving gives back the same policy.

Where it comes up