Paul's Python CoursePPC

Grid search

Not met yet

GridSearchCV(model, grid, cv=5) cross-validates every combination of settings in grid and keeps the best. The grid is a dict from a setting's name to the values to try: two settings with three and four values make twelve combinations, each fitted once per fold. Inside a pipeline, a setting is named for its step, two underscores, then the setting: kneighborsclassifier__n_neighbors. After fit, best_params_ holds the winner and best_score_ its mean score, and the winner is refitted on all the data, so the search can predict. best_score_ was chosen for being highest, so it flatters: report a test score.

Try it

main.py

Output

Press Run and what your program prints shows up here.

Easy to get wrong

  • Thinks grid search fits the model once per setting, rather than once per setting for every fold.
  • Thinks grid search ends with one of its fold models, rather than refitting the best setting on all the data.
  • Thinks a setting inside a pipeline is named as on the bare model, rather than with its step's name and two underscores.
  • Thinks a grid search's best cross-validated score is an honest estimate of how the model will do on new data.

Where it comes up