96. Bad Supreme Court Math

A recent effort to defend the Fifth Circuit's SCOTUS track record usefully (if unintentionally) highlights several of the most common methodological errors in efforts to marshal Supreme Court data

Welcome back to “One First,” a weekly newsletter that aims to make the U.S. Supreme Court more accessible to all of us. 

Every Monday, I’ll be offering an update on goings-on at the Court (“On the Docket”); a longer introduction to some feature of the Court’s history, current work, or key players (“The One First ‘Long Read’”); and some Court-related trivia. If you’re enjoying the newsletter, I hope you’ll consider sharing it (and subscribing if you don’t already): 

Although the Court made a little bit of news last week (more on that below), I wanted to use this week’s issue to address a piece published by Carrie Severino in the National Review on Friday—titled “The Left’s Lies About the Fifth Circuit’s Reversal Rate.” I’m not usually inclined to engage with Severino (or the NR more generally). But given that I’m one of the people Severino accuses of lying, it seemed worth summarizing exactly what her claim is and then explaining, in detail, why she both (1) completely misstates what I’ve previously written; and (2) makes some pretty egregious data-driven mistakes of her own. The former may help, if nothing else, to correct the record; the latter may be useful more generally because it underscores the dangers of superficial attempts to use data when trying to describe the Supreme Court’s output.

But first, the news.

---

On the Docket

The biggest (and just about only) news out of the Court last week was Thursday’s order granting in part and denying in part the Republican National Committee’s emergency application seeking to put back into effect three different Arizona laws relating to voting and documentary proof of citizenship: requirements that (1) those registering to vote using Arizona’s state paper form provide such proof; (2) those seeking to vote by mail provide such proof; and (3) those voting for the President provide such proof.

Over public dissents from Justices Sotomayor, Kagan, Barrett, and Jackson, one majority (Chief Justice Roberts and Justices Thomas, Alito, Gorsuch, and Kavanaugh) granted the RNC’s application as applied to (1)—the registration requirement. But a different majority (Sotomayor, Kagan, Barrett, Jackson, and one/both of the Chief Justice and Justice Kavanaugh)¹ denied the application as applied to (2) and (3)—the voting requirements:

There’s a lot that’s striking here. First, it’s rather remarkable that Thomas, Alito, and Gorsuch would’ve put all three laws back into effect—a move that, had it applied to recent registrations, might have prevented a large number of Arizonans (especially, as I understand it, college students) who are legally entitled to vote (and duly registered) from casting mail-in and/or presidential ballots in the upcoming election. A big enough possibility in the abstract, it’s only exacerbated by Arizona’s status as a critical state both in the presidential election and with regard to control of the Senate. It seems to me, if you’re going to vote for a result that could produce substantial disenfranchisement of eligible voters with such potential effects, you should at least explain why that result is legally required.

Second, here’s yet another high-profile emergency application where Justice Barrett is publicly distancing herself from all of the other Republican appointees—much like Ohio v. EPA, where she not only dissented, but wrote on behalf of the dissenters. I wrote a piece in early July about Barrett’s increasingly fascinating—and important—independence. Here’s yet another significant example.

Third, and most importantly, there’s the difference-splitting votes of one/both of the Chief Justice and Justice Kavanaugh—and the difficulty of reconciling those votes with the Court’s purported adherence to the controversial “Purcell principle.” Professor Rick Hasen wrote about this over at his Election Law Blog on Friday, but to make a long story short, the best defense of Purcell is the need to avoid court rulings that increase voter confusion heading into an election cycle. How can that “principle” be reconciled with a ruling that puts back into effect Arizona’s requirement that those registering to vote on the state paper form provide documentary proof of citizenship, when those who register online or use the federal form don’t have to? Or the need for different ballots depending upon whether the voter registered with a state paper form, a state electronic form, or a federal form? Or the possibility that those who used the state paper form while the proof-of-citizenship requirement was blocked might now have their registrations rejected?

If any/all of that sounds like it’s potentially confusing, it’s confusion created largely by the Supreme Court’s ruling on Thursday—since the provision had otherwise been frozen by a federal court order issued … last year. And although there may be arguments for why such a result is consistent with Purcell (I’m skeptical), or why Purcell just doesn’t apply in this case (that persuasively explains why it applies in all of the other ones), the majority in the Arizona case . . . didn’t provide them.

if the Court (and, especially, Justice Kavanaugh—who is perhaps the most frequent public defender of Purcell) is going to justify numerous earlier interventions to undo lower-court rulings in election-year disputes on the ground that the courts’ rulings risk confusing voters, it seems like a ruling that could well cause significant confusion on the ground warranted at least a modicum of explanation.

Besides the expected housekeeping regular Order List last Monday, that was it for last week. That means the justices still have 20 pending emergency applications to resolve, almost all of which are now ripe—including nine in the EPA power-plant cases; two in the student-loan cases; and one about federal funding for abortion counseling. It’s impossible to predict which of those rulings (if any) will come down this week, but it’s a good bet that we’ll get at least one of them. Otherwise, the justices have nothing calendared until the next housekeeping Order List on Friday, September 6.

---

The One First “Long Read”:
SCOTUS Math is (Not That) Hard

Regular readers of this newsletter know that I pay a lot of attention to the Fifth Circuit—and its relationship with the Supreme Court. Some of that is a function of where, until July, I was living (i.e., in the Fifth Circuit); some of that is because the Fifth Circuit has become ground zero for just about every major lawsuit challenging anything with a partisan political valence. Thanks to both the ability to engage in judge shopping at the district-court level and the current composition of the New Orleans-based court of appeals, everyone from Republican state attorneys general to Elon Musk to right-wing anti-abortion groups is steering as much of their litigation work into Louisiana, Mississippi, and Texas as they can.

We saw the latest example on Friday—when Texas, on behalf of 16 red states and America First Legal, brought a new suit in Tyler, Texas challenging the Biden administration’s “parole in place” immigration policy. (This is the fourth lawsuit Texas has filed against the Biden administration in a geographically random Texas district court in the last month alone; the first three were all filed in Amarillo.) And I don’t think it’s an exaggeration to suggest that the Fifth Circuit had rewarded most of this behavior with a series of massively important rulings—the overwhelming majority of which have endorsed these challenges, even on novel (if not affirmatively bizarre) theories of why the plaintiffs had standing or why they were going to win on the merits.

This litigation behavior has led to a remarkable series of confrontations between the Fifth Circuit and the Supreme Court—where, in many of these cases, what the Fifth Circuit is doing has proven too extreme for even this Supreme Court majority. That was true last term (OT2022), and it was true to an even greater extent this term—in the mifepristone case (in which a 9-0 Supreme Court reversed the Fifth Circuit); the Rahimi Second Amendment case (8-1); the CFPB funding case (7-2); the social media jawboning case (6-3); and so on. Even the majority opinion in the fractured NetChoice ruling (about Florida’s and Texas’s efforts to regulate social media companies) took a series of not-so-subtle shots at the Fifth Circuit.

Based on that track record, I wrote a piece in July for The Atlantic, titled “The Fifth Circuit Won By Losing,” which attempted to document both how bad a term the Fifth Circuit had at the Court and how much it nevertheless succeeded—both in the three cases in which it was affirmed (each of which came in ideologically charged disputes) and, more generally, in moving the Overton Window with respect to the kinds of arguments that are now viable. Although the piece is behind a paywall, here’s the central claim it made:

Of the 11 appeals the justices heard from that court (itself an eye-popping total), the Fifth Circuit was reversed in eight of them—the most reversals, for the second year in a row, of any court in the country from which the Supreme Court took appeals. And many of those reversals were in some of the term’s most ideologically charged cases, such as lawsuits seeking to block access to mifepristone on a nationwide basis, to invalidate the way Congress funds the Consumer Financial Protection Bureau (and a host of other agencies), and to bar the Biden administration from even talking with social-media companies about public-health-related mis- and disinformation.

But for as bad a term as the Fifth Circuit would appear to have had, it still succeeded in shoving American law far to the right. First, even when the Fifth Circuit lost, it usually picked up at least one vote (and as many as three) from the justices, validating the non-frivolousness, even if not the correctness, of its extremist reasoning. Second, the losses have the effect of making the most radical Supreme Court in our lifetime appear to be more moderate than it in fact is—with the Court’s defenders seizing upon some of the reversals of the Fifth Circuit as proof that, despite a rash of controversial, ideologically divided rulings in other cases on everything from January 6 to environmental law to homelessness, the Court really is “surprising” in its moderation. Third, and most important, the Supreme Court still affirmed three of the Fifth Circuit’s outlier rulings—all in cases in which the three more liberal justices dissented. The Fifth Circuit lost a lot—and somehow it still won.

Enter, Carrie Severino, President of the “Judicial Crisis Network.” (A group about which … much has been written.) In a post for the National Review on Friday, Severino took issue with my Atlantic piece and a series of tweets from, among others, Elie Mystal and Mark Joseph Stern. As she wrote, “In its chastisement of the Fifth Circuit, the Left willfully ignores the broader universe of appeals to the high court, and presents a seriously distorted picture in the process.”

Here is the central passage from her piece:

As recorded by Empirical SCOTUS last month, the Fifth Circuit was reversed or vacated nearly 73 percent or two-thirds of the time — depending on whether cases consolidated for argument or decision are counted (eleven total) or excluded (nine total) in the tally — and affirmed in the balance of merits cases. If we add to the mix the Court’s decision last week not to grant a stay of a Louisiana district court’s preliminary injunction barring enforcement of the Biden administration’s Title IX rule, the Fifth Circuit reversal rate drops to two-thirds counting consolidated cases and 60 percent not counting them.

But those numbers do not reflect anything amounting to a singular rebuke of the Fifth Circuit. The Supreme Court usually reverses lower courts — this term, it did so in more than two-thirds of its appellate decisions. And consider that this term, seven or eight (depending on methodology) of the twelve other circuit courts faced higher reversal or lower affirmance rates than the Fifth.

• The First, Fourth, Sixth, Seventh, Tenth, and Federal Circuits were reversed or vacated 100 percent of the time, albeit each with a fraction of the number of merits cases that came from the Fifth Circuit. When counting consolidated cases, the Tenth Circuit is no longer in this category as it adds an affirmance.

• The Second Circuit was reversed or vacated 86 percent of the time, amounting to one affirmance and six reversals. Five of those six reversals were decided unanimously, including a noteworthy case involving the First Amendment consequences of government bullying of the NRA.

• The District of Columbia Circuit was never affirmed over the course of four cases — seven counting consolidated cases.

Thus, she argues, anyone who criticizes the Fifth Circuit but doesn’t mention its “wins” or criticize these other courts (“commentators who do not feign scholarly objectivity”) is engaged in a double standard.

There are three material problems with Severino’s take. One of them is descriptive; and two are methodological.

First, I can’t speak for her other targets on “the Left,” but I never claimed, in either The Atlantic piece or elsewhere, that the Fifth Circuit’s reversal rate was the highest. There will virtually always be a lower court with a 100% reversal rate—whether across one, two, three, or even four cases. My argument was different, as quoted above: that the Fifth Circuit had “the most reversals, for the second year in a row, of any court in the country from which the Supreme Court took appeals. And many of those reversals were in some of the term’s most ideologically charged cases.” Even as she accuses me of “lying,” Severino doesn’t actually dispute either the quantitative (“most reversals”) or qualitative (“some of the term’s most ideologically charged cases”) points that I made. Worse than that, the whole point of my Atlantic piece was how the terrible numerical track record obscured the ways in which the Fifth Circuit had succeeded (hence the title of my piece). She’s thus criticizing me not just for arguments that I didn’t make, but for not making the very point that the piece was devoted to making. Reading is hard.

Second, turning to methodology, Severino commits the precise error about which I’ve written in detail before—casually presenting Supreme Court data on the premise that all cases are equal, so that a unanimous reversal of the Federal Circuit in a patent case deserves the same weight as the unanimous reversal of the Fifth Circuit in the mifepristone case or the 5-4 reversal of the D.C. Circuit in Ohio v. EPA. The reason why this error is so infectious is because (1) it ignores the justices’ own role in choosing to decide these cases (of the 59 cases decided after argument during the current term, 58 were cases the Court chose to hear); and (2) it assumes that the pile of cases the justices are taking from each lower court are sufficiently comparable to permit these kinds of cross-circuit data-driven analyses—that the pools of cases have somehow been controlled for the relevant variables.

This is where the fact that Severino completely ignores how right-wing litigants are steering cases to the Fifth Circuit is especially damning; the cases coming from the New Orleans-based appeals court are not a random cross-section of appeals from across the country; they’re a carefully cultivated set of ideologically charged challenges to federal policies that were filed in the Fifth Circuit (and nowhere else) for a reason. That’s in striking contrast to Severino’s favorite target—the D.C. Circuit. She’s absolutely right that the D.C. Circuit was 0-4, all in big cases (Trump immunity; one of the two Chevron cases; the January 6 obstruction case; and Ohio v. EPA). But three of those cases had to be brought in the D.C. Circuit. Trump and Fischer were criminal cases where D.C. was the proper venue; Ohio was a case within the D.C. Circuit’s exclusive statutory jurisdiction. Thus, no one steered those cases to that particular court of appeals because of its ideological orientation.

That’s part of why the Fifth Circuit is such a big story—because it’s a story not just about the Fifth Circuit faring poorly, but about this Supreme Court’s growing (if incomplete) unwillingness to indulge litigants’ transparent efforts to use the Fifth Circuit to move American law to the right. If we’re going to talk honestly about the Fifth Circuit’s track record, we have to talk about that, too. At the end of the day, that’s the significance of the Fifth Circuit’s poor track record.

Third, my favorite part of Severino’s piece is when she throws in the justices’ recent ruling on the Biden administration’s emergency applications in the Title IX cases as another “win” that should count toward the Fifth Circuit’s total statistics—cherry-picking one of the full Court’s 98 rulings on emergency applications thus far this term because it happens to support her thesis. In her words, “If we add to the mix the Court’s decision last week not to grant a stay of a Louisiana district court’s preliminary injunction barring enforcement of the Biden administration’s Title IX rule, the Fifth Circuit reversal rate drops to two-thirds counting consolidated cases and 60 percent not counting them.”

Well actually, if we “add to the mix” all of the Court’s decisions on emergency applications this term, it makes the Fifth Circuit look even worse. Of the 15 grants of emergency relief so far this term, seven of them came in cases originating in the Fifth Circuit—where the justices were providing emergency relief that the lower courts refused to provide, or wiping away emergency relief that they provided. In other words, 46.7% of the emergency relief the Supreme Court has issued this term has been directed to the same lower courts—those within the Fifth Circuit. And that’s not counting the SB4 case—in which Justice Barrett’s opinion concurring in the denial of emergency relief effectively forced the Fifth Circuit to grant that relief itself.

In the interest of completeness: Yes, the Court has also granted four applications in cases from the D.C. Circuit—but that was all in the same consolidated dispute (Ohio v. EPA). And yes, the Court has denied a bunch of applications from the Fifth Circuit, including in at least some ideologically charged cases. But no other court has an emergency-application numerator like the Fifth Circuit’s. So if we’re going to make claims respecting how rulings on emergency applications affect narratives about a lower court’s relationship with the Supreme Court, we ought to account for the entire dataset—and not just pick one recent example that happens to support our claim.

I don’t know if Severino just didn’t read what she was purporting to criticize, or if she did. But it seems like we all ought to be more careful before we accuse anyone of “lying” about something related to the Supreme Court—especially when our accusations are based upon descriptively and methodologically flawed accounts of our own.

---

SCOTUS Trivia:
100 Full-Court Decisions on Emergency Applications

Speaking of data (and as briefly noted above), Thursday’s ruling on the RNC’s emergency application in the Arizona proof-of-citizenship case was the 98th full-Court ruling on an application during the October 2023 Term (which, as of today, has six more weeks to go). It’s entirely possible that we’ll cross the century mark sometime this week—which is a striking milestone. I only have complete data for the last five terms (I’m working on it!), but here are the total numbers of full-Court rulings on applications at least going back to OT2019 (the COVID year):

OT2023: 98 (and counting)
OT2022: 76
OT2021: 72
OT2020: 66
OT2019: 72

My anecdotal sense is that there was a term or two in the mid-1980s when the Court was resolving similar numbers of emergency applications—the vast majority of which were in capital cases. But more than 100 full-Court decisions on emergency applications in a single term is certainly not something we’ve seen since then—and not something we’d ever seen before then, either. (Until 1980, most emergency applications were resolved by individual justices.) By itself, that number is just trivia. But it’s trivia that speaks to a broader (and important) pattern.

---

I hope that you’ve enjoyed this installment of “One First.” If you have feedback about today’s issue, or thoughts about future topics, please feel free to e-mail me. And if you liked it, please help spread the word!:

If you’re not already a paid subscriber and are interested in receiving regular bonus content (or, at the very least, in supporting the work that goes into this newsletter), please consider becoming one! This week’s bonus issue for paid subscribers will drop on Thursday. And we’ll be back with our regular content for everyone next Monday!

Happy Monday, everyone. I hope that you have a great week!!

1. It’s reasonable to think it was both of them, and that the vote was 6-3 as to the second and third requirements. But we don’t know for sure. All that we can say with certainty is that at least one of the Chief and Kavanaugh voted that way—because it would’ve taken a fifth vote to deny the application as applied to those requirements, and because we know that vote didn’t come from Thomas, Alito, or Gorsuch (who noted that they would’ve granted the applications in full).