Paul's Python CoursePPC

Baselines

Not met yet

A baseline is the simplest forecast that could possibly work, scored on exactly the same days as the model: always the commonest answer in the training data, tomorrow the same as today, or the rule the old hands use. Accuracy means nothing until you know what a baseline scores. When good days are rare, always saying no can be right most of the time, and a model that only matches it has learnt nothing. Take the commonest answer from the training data, never the test set. A model earns its place by clearly beating every sensible baseline on data it has never seen.

Try it

main.py

Output

Press Run and what your program prints shows up here.

Easy to get wrong

  • Thinks the baseline for a yes-or-no forecast is always a coin toss, 50%, whatever the answers' shares.
  • Thinks a high accuracy shows a model is good without comparing it with a simple baseline on the same days.
  • Thinks the commonest-answer baseline may take its commonest answer from the test set.
  • Thinks a model and a baseline can be compared fairly on different days.

Where it comes up