Pop quiz: which boys' team has had the best record of improvement from year to year? It's one of these three schools - but the answer depends on how you look at the data.
I started the Preseason Countdown in 2011, but in 2014 I added a new wrinkle: Improvement Ratings. This is a custom metric that I developed by pulling key data from the NCRunners database and then manipulating it in a spreadsheet. The goal of the Improvement Rating is to measure the year-to-year progress of a program, which helps in predicting which teams will improve the most from one cross country season to the next. Summer training, feeder programs, transfers, coaching changes - all of these will be reflected one way or the other in this statistic.
I've now gathered 7 years of data on over 50 of the best programs in the state. We can analyze that data for interesting trends, and the Improvement Rating seems to have good predictive value when it comes to the upcoming season. I've also added a new feature this year: in addition to an overall Improvement Rating, I've created a "Size-Adjusted" version of the metric, which uses classification rankings instead of overall rankings to determine how a program improves compared to those of similar size.
How It Works
The premise is simple: I wanted to calculate how much a team improves from the end of one cross country season to the next. I accomplished this by taking a team's returning 5-runner average time from the end of one season, and comparing it to the actual 5-runner average time from the end of the next season. In short, I compare a team's returning strength going into the season with their final result at the end of the season. The difference between them is the key: I subtract the two, with "positive" improvement meaning a drop from returning to actual, and "negative" improvement meaning an increase from returning to actual (which does happen).
I also wanted to adjust the raw improvement numbers to account for how highly-ranked the team is. In my experience, it's a lot harder for an elite team, with several runners that are already highly trained, to improve as much as a lower-ranked team whose athletes have not been through as much training. I used a team's final all-classes ranking for this part of the calculation, which allows me to adjust for late-season injuries because it uses the best times from the entire season.
Once I have the two factors (improvement and final ranking) for each individual season, I calculate 3-year, 5-year, and 7-year averages for each of them. The final Improvement Rating is calculated by dividing average improvement by the square root of the average ranking for each time period (the square root helps fix a problem I had with scattered data).
Overall vs. Size-Adjusted
The only problem I've ever had with the original Improvement Rating is the wide range of final rankings that comes from using all-classes rankings (teams can be well up in the 100's if they had a bad season). This tends to create a relatively low mean with a small number of teams that have very high improvement ratings (in other words, most teams are clustered near the average, but the top-ranked teams look like outliers, often several standard deviations above the mean). Using the square root of final ranking helped, putting the data into a semblance of a bell curve, but I've always been a bit dissatisfied with that solution.
This year, I had the idea to use each team's finish in its respective state meet. This adjusts for the size of the school by comparing its improvement to other schools in the same classification. On the other hand, this approach has two drawbacks: it leaves out schools from larger classifications that don't make the state meet but might still be in the top 25, and it doesn't account for late-season injuries that hurt a team's standing at the state meet. The first can be be corrected, but the second just has to be understood - there is an element of potential "luck" involved whether we look at overall or size-adjusted ratings (injuries, transfers, etc), and we just have to use a critical eye to look for potentially misleading results.
With all that methodology explained, check out the top 10 teams in each of the following time periods: