groupby, aggregation and pivot tables
Not met yet
census.groupby("year") sorts the rows into one group per year. Pick a column and say what to work out, ["age"].mean(), .size() or ["household"].nunique(), and you get one answer per group, labelled by the group and sorted. Group by a list of columns for one group per combination, and use .agg(people=("name", "size"), oldest=("age", "max")) for several named answers at once. pivot_table(index="parish", columns="year", aggfunc="size", fill_value=0) lays the answers out as a grid. size() counts rows; count() counts only values that aren't missing.
Try it
main.pyOutput
Press Run and what your program prints shows up here.
Easy to get wrong
- Thinks groupby lists the groups in the order they first appear. It sorts them by their keys unless told sort=False.
- Thinks groupby followed by an aggregation like mean() gives one answer per original row, rather than one per group.
- Thinks count() and size() are the same. size() counts the rows in each group; count() counts only the values that aren't missing.
- Thinks a number that restarts in each year or batch, like a household number, picks out one thing on its own, so grouping by it alone lumps different things together.
- Thinks the average of the group averages is the overall average. It is only when every group is the same size.