Perhaps the easiest way to start a fight with NFL fans is to assert that their favorite franchise hasn’t beaten anyone. In one fell swoop you’ve devalued a season’s worth of victories and also cast substantial doubt on the team’s postseason prospects. Accusations of a cupcake schedule make for sensitive discourse because a handful of teams each year actually do get lucky and face a slate of pushovers. And the idea of winning with a soft schedule is a kind of fraudulent, stolen glory that is almost too terrible for fans to contemplate. But if there are charlatan teams out there, how can we best identify them?

Quantifying strength of schedule can be tricky, and there are numerous approaches to the problem. Luckily two of the better metrics are in general agreement. Despite coming at the problem from different directions, the schedule-strength rankings at Pro-Football-Reference.com and Pro Football Focus correlate highly with each other this year. This means we can be reasonably confident that, for instance, the Cincinnati Bengals and Philadelphia Eagles did indeed face easier opponents over the course of 2021 than the Washington Football Team and the New York Giants did.

It’s notable that both metrics indicate that teams with easier schedules tended to make the playoffs this year. Among the teams in the top 10 hardest schedules of both systems, only the Kansas City Chiefs have so far made the playoffs. That should bode well for their playoff run, since K.C. is battle-tested and managed to win 11 games despite the tough slate. Conversely, getting by on wins versus inferior opponents might spell playoff disaster for teams like New England or Buffalo.

To get a clearer picture, though, we need to dig a bit deeper. Quarterback strength of schedule offers an alternate way of approaching the problem. The rationale for a QB-centric SoS analysis is straightforward: The quarterback is the most important player on the team, and his availability and performance moves lines in the betting markets. Given this, it’s useful to drill down into the position to better understand just how tough (or easy) a team’s schedule was. 

It turns out that teams that face a slate of bad QBs tend to win more games. While that’s not surprising, it’s comforting to verify that our assumptions about the importance of QB play are sound. If we build a simple model using data from 2006 through Week 17 of 2021 to predict a team’s regular-season win percentage and use Total Quarterback Rating of the opponent as the only predictor, we can generate an estimate of what a team’s win percentage should be. We can then use the model to identify teams that have underperformed relative to the strength of their opposing QB schedule.

Using this approach, the biggest disappointment in the NFL this year is the Buffalo Bills. With the regular season nearly over, in 2021 the Bills faced the worst collection of quarterbacks by Total QBR since at least 2006, our first year with QBR data. Yet despite having the great fortune to face a historically weak set of opposing QBs, Buffalo didn’t turn its advantage into wins at the rate we would expect. Given its opponent QBR of 25.4 through Week 17, its win percentage of 62.5 is 15.8 percentage points lower than expected — the biggest dropoff among teams that have clinched playoff spots.

One potential critique of opponent QBR is that it’s also capturing a team’s defensive performance. The Bills may have seen poor QB play, but perhaps they also created it through great defense. And while it’s true that the Bills defense has been good this season, a brief look at the list of quarterbacks the Bills played this year is instructive. 

The Bills started the year against the soon-to-be-retired Ben Roethlisberger and followed that loss with games against three second-string QBs: Jacoby Brissett, Taylor Heinicke and Davis Mills. Later in the season, they played rookie Trevor Lawrence and the two-win Jaguars, backup Mike White of the four-win New York Jets, New Orleans backup Trevor Siemian (who made his last start of the season against Buffalo), a game against New England rookie Mac Jones in which he attempted just three passes for 19 yards, and a late-career Cam Newton trotting out the worst passing offense in the league. The Bills defense may be good, but the bad emanating from this collection of pigskin tossers is unmistakable.

It’s also true that when the Bills did face higher-caliber QBs, they didn’t fare too well, going 2-3 against the five best QBs they faced in 2021. The Bills did a masterful job beating Patrick Mahomes and the Chiefs by playing Cover 2 on over half their defensive snaps, and they dispatched Matt Ryan and the Falcons in Week 17. But they lost games to Ryan Tannehill, Carson Wentz and defending Super Bowl champion Tom Brady.

The Bills defense held opposing QBs under the quarterback’s seasonlong Total QBR nine times, including the terrific defensive performance against Mahomes and the Chiefs and in a game against the Texans that saw an 87-yard, four-interception stinker from Mills. But in seven games, they allowed an above-average performance from the opposing QB relative to his seasonlong Total QBR, including Tua Tagovailoa’s second-best performance of the season by QBR.

Buffalo is in the tournament, and anything can happen — but it underperformed in a season that was perfectly set up for it to dominate. The Bills played against five backup QBs and two tossers who are somewhere between certainly and highly likely to retire. No one can take away the Bills’ 10 wins thus far, but we should probably be skeptical of their chances of success in the playoffs. It’s not at all unfair to say that of all the teams in the postseason, the Bills are the team that’s done the least with the most, and that will have to change if they have any hope of hoisting the Lombardi Trophy. 

Check out our latest NFL predictions.