Paul's Python CoursePPC

k nearest neighbours

Not met yet

k nearest neighbours learns no weights. It keeps every past case, and to answer a new one it measures the distance to each, takes the k nearest, and lets them decide: a vote for a label, a mean for a number. Straight-line distance is np.sqrt(((X - point) ** 2).sum(axis=1)), one per row; Manhattan distance adds the plain differences instead. Scale the readings first, with the training data's means and spreads, or the reading with the biggest numbers decides alone. np.argsort gives the positions nearest first. k = 1 is perfect on its own training data, because every case is its own nearest neighbour, and that proves nothing: choose k on validation data, where small k chases odd cases and large k drowns rare ones.

Try it

main.py

Output

Press Run and what your program prints shows up here.

Easy to get wrong

  • Thinks the distance between two sets of readings weighs every reading fairly, whatever its units, so a reading counted in hundreds can't swamp one counted in tenths.
  • Thinks k nearest neighbours learns weights while training, rather than keeping every past case and measuring distances when asked.
  • Thinks the perfect training accuracy of k = 1 shows it forecasts well, when every training case is its own nearest neighbour.
  • Thinks a larger k always forecasts better because more neighbours means more evidence, when a large k drowns rare answers in the commonest one.
  • Thinks np.argsort gives the distances in order, rather than the positions of the rows in order of distance.
  • Thinks .sum(axis=0) adds along each row, when it adds down each column and gives one total per reading.

Where it comes up