Paul's Python CoursePPC

Gymnasium environments

Not met yet

Gymnasium is the standard shape for reinforcement learning environments. reset(seed=...) starts an episode and returns (observation, info); step(action) returns (observation, reward, terminated, truncated, info). terminated means the episode ended in its own right (a goal, a fall, a flood); truncated means it was stopped from outside, usually by a time limit, so the last state still has a future worth counting. action_space and observation_space (Discrete, Box) say what's allowed. A custom environment subclasses gym.Env, calls super().reset(seed=seed), and draws every random number from self.np_random, so a seed repeats a run exactly.

Try it

main.py

Output

Press Run and what your program prints shows up here.

Easy to get wrong

  • Treats an episode stopped by a time limit as if it ended in its own right, setting terminated rather than truncated.
  • Writes the episode loop to stop only when terminated, so an episode that is only ever truncated never stops.
  • Expects step to return four things with a single done flag, as the old Gym API did, rather than Gymnasium's five.
  • Thinks reset returns just the observation, forgetting the info dictionary that comes with it.
  • Thinks reset() without a seed starts the same random run again, when it carries on with the generator the last seed made.
  • Leaves out the value of the next state when an episode is truncated, as if a time limit meant nothing could follow.

Where it comes up