Tweet
As many of you know, from time to time I pull a slew of data out of FOF and do a bit of data mining. You can find some past attempts and a better description of the methods here: (NAFL) and here: (RNFL).
This is s re-analysis of the connection between quarterback ratings and quarterback performance now that we are in the 6.3 era.
The techniques are the same as the first time, but there are a few twists. It is still a stepwise regression to determine how bars relate to stats; so, a general linear model in which the statistics are assumed to depend linearly on bars and some noise (i.e. all the other factors that aren't modeled which should have a huge impact - o-line, receivers, etc.). The linearity assumption is almost certainly wrong but a first-order approximation is still informative.
The twists are that I reran each season 5 times. So, these data cover a span of many seasons with 5 samples of each season. In the end I just lumped this together, but I like having many of the samples come from very similar conditions.
A note about reporting. In the recent studies I've reported p-vales, correlation coefficients, and convoluted metrics. This time, I'll just report the correlation coefficients. These tell us how much improvement we can expect in a stat from increasing the bar by one point.
I'll go in the same order as before.
Completion Percentage:
Accuracy: 0.029
To put this in perspective, this says the difference between a passer with 50 accuracy and one with 100 is only about 1.45%. That's a nice improvement, but it leaves a lot of variance unexplained.
The pre-6.3 analysis found a significant correlation only with short passes. Seeing accuracy impact this stat forces me to think completely differently about how the ability is utilized. I always thought of it as "hitting the receiver in stride" and therefore getting more YAC. This correlation seems to imply that it is also used in the way we think it would impact real football.
Average per Catch:
Accuracy: 0.015
Short Passes: -0.009
Scramble Frequency: -0.005
We have the same short passes penalty (notice the negative signs) as the pre-6.3 analysis, but now accuracy is the dominating positive; likely because of the increase importance of YAC. Medium passing and sense rush were important before, but it seems that longer passes are no longer the best way to get a high YPC.
I'm guessing scramble frequency has an impact because you are more likely to bail on longer routes when the pressure is coming.
Average per Attempt:
Accuracy: 0.012
Short Passes: -0.005
Scramble Frequency: -0.003
This is just AVC all over again but with a lesser impact from each. This make total sense. I would not have been surprised to see the impact of accuracy increase, though, since it impacts completion percentage.
Touchdowns per Completion:
Accuracy: 0.0002
Meh…. I'm not even sure why I included this one, last time. It makes an interesting comparison, though. Many factors showed up in this pre-6.3. I think that they don't now because factors other than QB bars have a bigger impact. Accuracy showed up prominently in the last one, as well.
Interceptions per Attempt:
Height: -0.002
Short Passes: -0.0002
Huh?
The results from the last test were very noisy (high p-values) for this stat. These two are very solid (unlikely the result of noise). Make of it what you will.
My only explanation is that the short passing stat causes more short pass attempts? I don't have the data to test this hypothesis. It could also be that there are so many more short passes attempted by the CPU now, that the bar comes into play more often.
Height is beyond me, but I have a vague memory of someone saying it plays into short passing more, so the above would apply, again.
Twenty+ Throws per Catch:
Timing: 0.0002
Scramble Frequency: -0.0002
I'm guessing that "taking advantage of defensive miscues" also means to catch a blown assignment in the secondary and therefor score a big play? If so, I think this is awesome. I just wish I knew what caused the defense to do this. Play diagnosis? Coverage ratings? Maybe we will find out in a further analysis.
Scramble would be explained in the same way as for YPC. If you bail on deep routes more often, then you'll have less deep completions… *shrug*
Passer Rating:
Accuracy: 0.121
Kerpow.
Well, not too surprising, I suppose. If you de-emphasize sense rush and muddle the waters (i.e. make non-qb bar factors more important) to remove the factors from per-6.3 with small correlations then you end up with just accuracy.
It's interesting that it explains over 8% of the variance in passer rating though. That's a lot.
I've included a few extra stats, this time.
3rd-Down Conversion:
Long Passes: 0.114
Third-Down Passing: 0.058
Scramble Frequency: -0.056
Scramble frequency makes sense. The dumb-ass QB is bailing on the route that would get you a first down. 3D passing is more of a sanity check then anything else. If it didn't show up there, I would be worried. Long passes is surprising. I would have expected medium passing since those are the important route lengths usually (11 to 20) for converting. *shrug*
It's worth noting that 3rd-Down Completion % plays out the same but without the long-passes correlation.
3rd-Down Conversion %:
Accuracy: 0.074
Timing: -0.024
I'm at a loss. Accuracy makes sense, but timing? Maybe a high-timing QB too often notices a good downfield opportunity (which is lower %) and therefore goes for the big play rather than the 5 yards needed for a first down? No idea.
Sacks per Attempt:
Sense Rush: -0.0005
Accuracy: -0.0001
Read Defense: 0.0003
Now, these are small numbers, but remember, this is sacks per attempt which is also a very small number, so these are high. Very high, in fact; these three stats explain 25% of the variance in the data.
At first I was put off by this, but an explanation does present itself for read defense, at least. Notice that read defense increases the number of sacks per attempt. But, read defense is your QBs ability to go through his progression if the first receiver is covered. That means he holds the ball longer. If this is the explanation, then I think it's terrific that this is in the game.
I haven't the vaguest clue why accuracy would help avoid sacks per attempt. I don't think it has anything to do with sacks directly, but rather with the types of situations high-accuracy versus low-accuracy qbs find themselves in.
Happy debating.
This is s re-analysis of the connection between quarterback ratings and quarterback performance now that we are in the 6.3 era.
The techniques are the same as the first time, but there are a few twists. It is still a stepwise regression to determine how bars relate to stats; so, a general linear model in which the statistics are assumed to depend linearly on bars and some noise (i.e. all the other factors that aren't modeled which should have a huge impact - o-line, receivers, etc.). The linearity assumption is almost certainly wrong but a first-order approximation is still informative.
The twists are that I reran each season 5 times. So, these data cover a span of many seasons with 5 samples of each season. In the end I just lumped this together, but I like having many of the samples come from very similar conditions.
A note about reporting. In the recent studies I've reported p-vales, correlation coefficients, and convoluted metrics. This time, I'll just report the correlation coefficients. These tell us how much improvement we can expect in a stat from increasing the bar by one point.
I'll go in the same order as before.
Completion Percentage:
Accuracy: 0.029
To put this in perspective, this says the difference between a passer with 50 accuracy and one with 100 is only about 1.45%. That's a nice improvement, but it leaves a lot of variance unexplained.
The pre-6.3 analysis found a significant correlation only with short passes. Seeing accuracy impact this stat forces me to think completely differently about how the ability is utilized. I always thought of it as "hitting the receiver in stride" and therefore getting more YAC. This correlation seems to imply that it is also used in the way we think it would impact real football.
Average per Catch:
Accuracy: 0.015
Short Passes: -0.009
Scramble Frequency: -0.005
We have the same short passes penalty (notice the negative signs) as the pre-6.3 analysis, but now accuracy is the dominating positive; likely because of the increase importance of YAC. Medium passing and sense rush were important before, but it seems that longer passes are no longer the best way to get a high YPC.
I'm guessing scramble frequency has an impact because you are more likely to bail on longer routes when the pressure is coming.
Average per Attempt:
Accuracy: 0.012
Short Passes: -0.005
Scramble Frequency: -0.003
This is just AVC all over again but with a lesser impact from each. This make total sense. I would not have been surprised to see the impact of accuracy increase, though, since it impacts completion percentage.
Touchdowns per Completion:
Accuracy: 0.0002
Meh…. I'm not even sure why I included this one, last time. It makes an interesting comparison, though. Many factors showed up in this pre-6.3. I think that they don't now because factors other than QB bars have a bigger impact. Accuracy showed up prominently in the last one, as well.
Interceptions per Attempt:
Height: -0.002
Short Passes: -0.0002
Huh?
The results from the last test were very noisy (high p-values) for this stat. These two are very solid (unlikely the result of noise). Make of it what you will.
My only explanation is that the short passing stat causes more short pass attempts? I don't have the data to test this hypothesis. It could also be that there are so many more short passes attempted by the CPU now, that the bar comes into play more often.
Height is beyond me, but I have a vague memory of someone saying it plays into short passing more, so the above would apply, again.
Twenty+ Throws per Catch:
Timing: 0.0002
Scramble Frequency: -0.0002
I'm guessing that "taking advantage of defensive miscues" also means to catch a blown assignment in the secondary and therefor score a big play? If so, I think this is awesome. I just wish I knew what caused the defense to do this. Play diagnosis? Coverage ratings? Maybe we will find out in a further analysis.
Scramble would be explained in the same way as for YPC. If you bail on deep routes more often, then you'll have less deep completions… *shrug*
Passer Rating:
Accuracy: 0.121
Kerpow.
Well, not too surprising, I suppose. If you de-emphasize sense rush and muddle the waters (i.e. make non-qb bar factors more important) to remove the factors from per-6.3 with small correlations then you end up with just accuracy.
It's interesting that it explains over 8% of the variance in passer rating though. That's a lot.
I've included a few extra stats, this time.
3rd-Down Conversion:
Long Passes: 0.114
Third-Down Passing: 0.058
Scramble Frequency: -0.056
Scramble frequency makes sense. The dumb-ass QB is bailing on the route that would get you a first down. 3D passing is more of a sanity check then anything else. If it didn't show up there, I would be worried. Long passes is surprising. I would have expected medium passing since those are the important route lengths usually (11 to 20) for converting. *shrug*
It's worth noting that 3rd-Down Completion % plays out the same but without the long-passes correlation.
3rd-Down Conversion %:
Accuracy: 0.074
Timing: -0.024
I'm at a loss. Accuracy makes sense, but timing? Maybe a high-timing QB too often notices a good downfield opportunity (which is lower %) and therefore goes for the big play rather than the 5 yards needed for a first down? No idea.
Sacks per Attempt:
Sense Rush: -0.0005
Accuracy: -0.0001
Read Defense: 0.0003
Now, these are small numbers, but remember, this is sacks per attempt which is also a very small number, so these are high. Very high, in fact; these three stats explain 25% of the variance in the data.
At first I was put off by this, but an explanation does present itself for read defense, at least. Notice that read defense increases the number of sacks per attempt. But, read defense is your QBs ability to go through his progression if the first receiver is covered. That means he holds the ball longer. If this is the explanation, then I think it's terrific that this is in the game.
I haven't the vaguest clue why accuracy would help avoid sacks per attempt. I don't think it has anything to do with sacks directly, but rather with the types of situations high-accuracy versus low-accuracy qbs find themselves in.
Happy debating.
Comment