Why On Net Percentage is Real and Why it Matters

In today’s SI.com piece (http://www.si.com/nhl/2016/02/05/advanced-hockey-stats-shot-attempts-value) we decided to wade into the current controversy over whether or not generating higher quality shots is a repeatable skill.

To further complicate matters we decided to look for something after the fact – in effect indirect evidence that certain teams are generating higher quality shots – rather than identify the specific action or game situation that makes one team’s shots consistently better than another’s.

There’s a lot to be said on the topic, and we’re not going to pretend to have come anywhere close to addressing it fully. That will require years of effort with insights shared between lots of smart people and of course tracking technology to give us much more reliable data.

However, we do think the shot attempt has become so fetishized that at least some analysts are assuming all hockey dogma is pure nonsense and they wind up confusing skill with luck simply because it doesn’t fit their model.

When we looked at one bit of basic hockey dogma – get your shots on net – we were equally open to the possibility that at best we might find that getting a higher percentage of shots through to goal was an element of unrepeateable “luck” as we were that we would find a repeatable skill that could be optimized.

As luck would have it (pun intended) we seem to have stumbled across a repeatable skill, one that functions at the team level no less.

We started by looking at every full NHL season since 2008-09 (six seasons excluding the lockout shortened season) and took the percentage of each team’s shot attempts that made it to the net (On Net %) up to January 15 and compared it to that team’s On Net % after January 15.

What we found is that there’s a strong relationship between a team’s On Net % in that first part of the season and the remainder of the regular season.

Specifically, the r-squared we generated was 0.26, which is a strong relationship and difficult to write off as luck! Of course the other 74% we’re not explaining here may be caused by lots of different factors (including injuries, differences in strength of schedule, coaching changes, etc.), but it’s difficult to write off the 26% as simply “luck”.

For those who like charts, the scatter plot is below.

On Net Graph 1

The next thing we looked at was the relationship between On Net % and True Sh%. In order to do that we simply paired each team’s On Net % during the relevant period and its True Sh% during the same period (giving us 360 observations in total). Here the relationship was less strong (r-squared of 0.13), but it was still meaningful.

On Net Graph 2

When we considered True Sh% up to January 15 and after, the relationship was quite poor, giving us an r-squared of only 0.038, only slightly better than team shooting percentage (0.036) and worse than team save percentage (0.052).

What’s clear from all of this is that while getting shot attempts on net is a repeatable skill at the team level and is correlated with scoring more goals, turning shot attempts into goals remains highly variable and subject to the vagaries of “puck luck”. Some aspects of this may be truly random and others skills nobody has managed to identify and quantify thus far. This fact applies whether you look at the NHL’s official shooting percentage or True Sh%, and it’s reason enough to be suspicious of a team that continues to score due to an unusually high team shooting percentage.

As noted in the SI.com piece, none of what we’ve found suggests that a team should start trying to optimize On Net % at the expense of other aspects of shot quality or total number of shot attempts (e.g. the advantage of getting a shot on net with traffic in front of the goalie will often outweigh the increased risk of that traffic preventing the shot from ever getting there).

But it does mean a team that’s consistently good at getting pucks on net (e.g. the Ducks) may be able to give up something in the possession battle and one that’s generating a lot of shot attempts but continuing to lose (e.g. the Canadiens) might want to look at their long-term On Net % rather than just imagine their puck luck will change or blame their goalie.

Introducing shot quality into the analytics conversation makes things a whole lot messier. But doing so makes sense of a complex and dynamic game. Like any such game, hockey demands more than a single strategy.