Manny Diaz, who's now making his second run at being defensive coordinator for Dan Mullen at Mississippi State, recently made a claim I don't recall hearing before in this exact form:
"We feel like we have to, especially this year. To win in our league, you have to (get turnovers)," Diaz said. "It’s always going to start with stopping the run. If you stop the run, you make them have to throw to beat you. If they have to throw to win, the ball is in harm’s way. No one turns the ball over more than the quarterback.
"There’s a bunch of things you can do to get after the quarterback to make him make mistakes. But if you can’t stop the run, then you have no chance of doing that. Our run defense will be the first thing we’ll pride ourselves on. Anything from that point on, that’s where the turnovers start to come."
It didn't take me long to realize this is basically a more verbose version of Robert Neyland's aphorism that, "when you throw the ball, three things can happen—and two of them are bad." It sounds different because Diaz is just speaking from a defensive perspective, rather than the offensive one where people use that quote to support a run-based offense. For a defensive coach, the fact that the pass play introduces the interception as a second possible turnover on top of the fumble is a feature, not a bug.
Still, I couldn't let this one slip by. This quote from Diaz sounds testable, so I wanted to see to what extent it's actually true. In doing so, I put all of the stats in terms of rates to avoid issues of pace mucking things up. A team in a conference where uptempo offenses rule might end up with more turnovers simply due to having more defensive plays and vice versa. Most will be familiar to you, but for turnovers I used turnover percentage: the total turnovers gained divided by total plays defended. As usual, I took sacks out of the run data, and when I talk about total run or total pass plays, sacks count as pass plays.
This isn't going to be totally accurate, because there isn't a stat source out there that separates turnovers on special teams from turnovers on standard plays. I believe that turnovers on special teams are sufficiently rare that the results here will be close enough, but just know that I understand the limitations here.
There are two ways to try to test Diaz's assertion that better run defense leads to more turnovers. The first is simply to see if having a better run defense leads to a higher turnover percentage.
I ran a correlation between yards per play allowed and turnover percentage using data from all I-A teams from the 2008-14 seasons. It came out to -0.310, meaning that as yards per play allowed goes down, turnover percentage goes up. This correlation, while not overly strong, does tend to back Diaz up.
What about pass defense, though? If pass defense has a stronger correlation with turnovers, wouldn't it make sense to focus more on that?
Normally I'd use passing efficiency defense for this, but passing efficiency includes interceptions as a part of the formula. It'd be surprising if an increasing number of interceptions didn't correlate with an increasing number of turnovers, right? So I used yards per attempt instead. It'll still feel some measure of effect from interceptions, as a pick will make yards per attempt go down, but that effect is far more muted with yards per attempt than with passing efficiency.
The correlation between yards per attempt allowed and turnover percentage is -0.396. It is true that pass defense better correlates with turnovers than run defense does. Is that a point against Diaz?
Maybe not. The other way to test this would be to see if there is a positive correlation with between the percentage of plays defended being passes and turnovers. After all, Diaz is saying he wants to get that pass percentage higher. The point of stopping the run, in Diaz's explanation here, is to force more passes.
As it turns out, there is a positive correlation between pass percentage and turnovers at 0.350. The more pass plays a team faces as a percentage of all plays defended, the more turnovers they get. That result is another one that boosts Diaz's assertion.
But wait a second. Let's apply a different truism, this one from Football Outsiders: "You run when you win, not win when you run." As Aaron Schatz explained it:
There are exceptions, usually when the opponent is strong in every area except run defense... [h]owever, in general, winning teams have a lot of carries because their running backs are running out the clock at the end of wins, not because they are running wild early in games.
Apply this to a defensive context, and winning teams will defend more passes than runs. Certainly it's possible to have a great defense that doesn't win a lot of games—see Auburn and Tennessee in 2008, or Florida in 2013—and it's possible to win a lot of games with a terrible defense—see 2011 Baylor, which won 10 games despite being 113th in scoring defense. There are always exceptions, and that's why these correlations are in the +/- 0.300 to 0.400 range rather than, say, the +/- 0.700 to 0.800 range.
Still, teams that win a lot of games usually have good defenses. We should also expect that good defenses will force a lot of turnovers. We're now stuck in the correlation vs. causation trap. Does strength at stopping the run cause a team to generate more turnovers? Or does simply being a good defense cause that unit to both stop the run and generate more turnovers?
It's certainly possible that Diaz has some kind of proprietary information that makes his case in a stronger way, as he's one of the few coaches out there who whole-heartedly embraces advanced analysis of the game.
One thing I can say for sure is this: when you share a division with Nick Saban, Bret Bielema, Gus Malzahn, and Les Miles, focusing on stopping the run isn't a bad plan.