Cashing in Big: A Look at Contract Year Effects

This week’s column in The Star was based on research done by Mark Drummond. Mark just graduated from the University of Waterloo as an economics major, and one of his graduation requirements was the writing of an undergraduate thesis. I was fortunate enough to act as supervisor to this project. Mark chose to examine the determinants of player salary in the NHL, in particular to see if GMs were overly influenced by the most recent year’s performance.

Mark started by using CapGeek to identify when a player’s contract expired. In order to examine how a player’s salary is affected by performance, Mark then went and categorized a variety of stats by when they occurred relative to the signing of a contract. The last year of a contract was labelled the “contract year” or CY, the year before that was CY-1, and so on. Table 1 shows the stats Mark was looking at, and the corresponding means, medians, and standard deviations.

A first glance didn’t suggest much was going on. At this point, it was already important to separate the players into UFA and RFA status, as there are very different processes that determine their salaries.

One thing that stood out was that RFAs generally get better as they get older, which is as one would expect given their young age. UFAs, on the other hand, were getting worse on average. This is, again, not too surprising, given that forwards tend to peak right around the time they transition from RFA to UFA, so their UFA years tend to be ones of decline.

Something that we did notice, however, was that the variance in performance was a bit higher for UFAs in their contract year. It’s not massively bigger, but it made us think that Mark should break players down into finer categories. There was another reason for this as well – high-end players presumably have greater bargaining power when it comes to negotiating their salary, and so it seemed worthwhile to make a distinction along those lines when looking at salary determination.

Mark decided to break players down into quartiles, which seemed logical. Four groups can separate top talent from pluggers while still leaving plenty of players in each group. That’s when we noticed the effects mentioned in The Star. As a recap, Table 2 shows how each quartile performs going into their contract years. That contract year sure stands out for the top quartile! And yes, the effect is statistically significant.

What Mark didn’t look at in his thesis (as it wasn’t relevant) was what happened when the new contract begins. Mark went and generated those data for The Star piece. Both of us were quite surprised to see that, after such a dramatic increase in the contract year, top players didn’t come right back down to earth the year following.

To be honest, it’s a little difficult to think about what could be causing this. It’s hard to imagine that the top players in the league are generally dogging it, and only turn it on when money is on the line. It is possible that players take their off-season regimen more seriously in their contract year, and that they don’t give up their good habits right away after cashing in. As far as explanations go, this one seems to have the greatest air of plausibility, at least in my opinion.

Finally, I should mention Mark’s main result. A player’s salary is, naturally, affected by the performance in all 3 seasons leading up to his new contract. However, the season that seems to have the biggest effect is not the contract year, but the year before. In other words, GMs don’t appear to be suckered in by that one outburst of production, at least on average. If anything, since the CY-1 year is, on average, the least productive year for top players, GMs are perhaps more cynical than they should be.

6 Comments

  1. Henry's Gravatar Henry
    November 11, 2014    

    How was performance measured to avoid look ahead and survivorship bias? Top quartile players are the ones that earn the big contracts, so of course they would display a boost in pts/game in CY and CY+1.

  2. Phil Curry's Gravatar Phil Curry
    November 11, 2014    

    Hi Henry,

    Your concern, as I understand it, is that we might be observing survivorship bias because there are players dropping out of the dataset in their contract year and the year after, and these players are below average for their quartile, thereby bringing the average up after they drop out.

    If I haven't got it right, let me know.

    But, there are no players dropping out in these data. We are only looking at players who signed a contract, and then compare the performance of those players in their contract year (the last year of their old contract), to the years leading up to it, and the year after.

    So, if there were 12 players who signed a contract in the data, we sorted them by how well they did in the last year of their old contract. The top 3 would then be in the top quartile. We then looked at how those 3 players did, on average, in the years leading up to the new contract, and in the first year of the new contract. So, there is no survivorship bias. Nobody drops out of the data - the quartiles contain the same players in the CY year as well as the CY+1, CY-1, etc years.

    I hope this answers your question. As I said, if I've got your question wrong somehow, let me know.

  3. henry's Gravatar henry
    November 11, 2014    

    By taking the pts/60 over the entire sample for each player, sorting them into quartiles, then looking at the YoY difference of each quartile, you are introducing look ahead bias. pts/60 should only include CY - 1 and before, not the entire sample.

    If you don't do this, you are merely looking at the YoY trend. Top quartile players may show an uptrend for reasons of age (entering their primes) and not related to CY effects at all. These other factors either need to be controlled for or performance should not be based on the entire sample.

  4. Phil Curry's Gravatar Phil Curry
    November 11, 2014    

    I'm not sure I'm following. First off, what do you mean by "the entire sample"? Do you mean all years, or all players? If you mean all years, then there's no problem, because we're not doing that.

    The Pts/60 measures are how that player did in that given year. So, from above, players in the top quartile average 2.4 pt/60 in their contract year and their contract year only, and those same players averaged 1.9 pts/60 the year before. These are not career averages - just averages across players for that year.

    There is no survivorship bias, because it is the same players being examined in all the years. There is no any look-ahead bias, because players are aware of when their contract year is, and we're only looking at performance in a specific year and seeing how it changes. You are correct, however, that we could be capturing something like an age profile. This is discussed in the article above. I'm sure we're capturing the effect of an age profile for RFAs. However, UFAs are generally past their peak. There is no way to reconcile a flat trend before the contract year and the spike up in the contract year with an age profile.

    Again, if I've misunderstood what you've said, let me know.

    • henry's Gravatar henry
      November 11, 2014    

      Thanks Phil. If the performance was taken for each year instead of across all years, then you and Mark have on your hands the most convincing evidence for contract year effect I've seen.

      • Phil Curry's Gravatar Phil Curry
        November 11, 2014    

        Thanks, Henry, that's a very kind comment. I'm glad we got that sorted out - and I apologize for any looseness in the writing that caused the confusion. But, I do have to admit that it really is Mark that deserves all the credit (and I deserve the blame for the write up here). I know he'll be happy to hear that you think so highly of his work!

Leave a Reply to henry Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>