Bonus 163: How (Not) To Count Supreme Court Rulings
As we enter the season of sweeping numerical claims about the Supreme Court's October 2024 Term, it's worth rehashing out the major flaws that are baked into (almost) all of those assertions.
Welcome back to the weekly bonus content for “One First.” Although Monday’s regular newsletter will remain free for as long as I’m able to do this, I put much of the weekly bonus issue behind a paywall as an added incentive for those who are willing and able to support the work that goes into putting this newsletter together every week. I’m grateful to those of you who are already paid subscribers, and I hope that those of you who aren’t will consider a paid subscription if and when your circumstances permit.
This week has brought with it the typical flood of “end-of-term” stories, podcasts, and public events. As my own participation in some of these suggests, I certainly think there is value in efforts to look at the work of the Court on a more holistic basis—and not just as the sum of a series of individual decisions, however worthy those decisions are of their own specific analyses. At the same time, I fear that many of those efforts are selectively holistic in ways that potentially bury some pretty important takeaways (or lead folks to make claims that aren’t really true—like “Justice Kagan was in the majority more than Justices Thomas, Alito, and Gorsuch”).
To some degree, this has always been a problem with these kinds of curtain-closing pieces. But I wanted to use today’s bonus issue to explain how the proliferation of emergency applications, in both quantity and quality, poses unique (and uniquely serious) challenges to how we measure the Court’s overall output. As I explain below the fold, although there’s no objectively clear answer to which rulings should and shouldn’t “count,” there’s also no question that many of the most visible end-of-term wrap-ups (with special apologies to SCOTUSblog) are leaving out rulings that they absolutely ought to be including—defects that, among other things, may make the Court appear to be less ideologically divided than what a more-complete account would suggest.
For those who aren’t paid subscribers, we’ll be back with our regular coverage of the Court (no later than) Monday. For those who are, please read on.
Problem #1: It Ain’t Over ‘Til It’s Over
Let’s start with the most obvious point: The Supreme Court’s October 2024 Term … isn’t over. Indeed, it won’t end until the stroke of midnight on Monday, October 6. And although the Court used to formally adjourn when it rose for its summer recess (that’s why, among other things, the Court’s docket numbers just reset to 25-n), it stopped doing that in 1980. The reality is that the Court remains open for business—everything up to, and including, oral arguments and full-blown rulings—all summer. (Indeed, we expect a batch of potentially significant orders in pending cases at 9:30 ET this morning.)
And as much as that’s been true for some time, it seems especially significant in a term in which we’ve already seen an unprecedented number of emergency applications—and an unprecedented number of rulings by the full Court on those filings. There’s every reason to believe that the Court is going to continue to be inundated with high-profile emergency applications over the next three months (the Trump administration just filed its 20th yesterday). It ought to follow that we expect some unpredictable number of significant rulings between now and October 6—rulings that may or may not be consistent with the trends we’ve seen over the first nine months.
Consider, in this regard, SCOTUSblog’s super-helpful “Stat Pack,” which purports to take a comprehensive look at all of the “opinions of the Court” issued during the October 2024 Term (more on my concern with focusing on this specific subset below). As the Stat Pack explicitly notes, it doesn’t include Monday’s opinion of the Court in Goldey v. Fields, a “per curiam” ruling that summarily reversed the Fourth Circuit,1 because, according to the Stat Pack’s authors, it came “after the term had already concluded.” Of course, that’s just not true. But it’s also inconsistent; by that logic, opinions of the Court handed down last summer should be included in this year’s “Stat Pack.” But you won’t find the Court’s August 16, 2024 “per curiam” decision in Department of Education v. Louisiana in this year’s Stat Pack.
As I explain below, I think the better practice is to wait for the term to be formally over. But at the very least, we ought to be consistent about how we’re counting rulings handed down during the summer recess—especially in a year in which we’re likely to get a bunch of really big ones.
Problem #2: Selectively Counting Rulings on Emergency Applications
The bigger problem, in my view, is that, as the Stat Pack reflects, a lot of folks are selectively counting rulings on emergency applications as part of the broader data set—sometimes not at all; and sometimes if, but only if, that ruling came with an “opinion of the Court” (whether that ruling is signed or “per curiam”). Thus, the SCOTUSblog Stat Pack includes Trump v. CASA, Inc. (the birthright citizenship case) and three other rulings on Trump-related emergency applications (Dep’t of Ed. v. California; Trump v. J.G.G.; and the second ruling in A.A.R.P. v. Trump). All four of those came with “opinions of the Court.” But it does not include the other 13 (or any of the numerous significant rulings so far this term on non-Trump-related emergency applications).
I imagine that the premise animating this distinction is the significance of the Court choosing to provide a written rationale for its intervention (or non-intervention). But that strikes me as a flawed justification for at least two reasons:
First, a number of the orders excluded from this dataset have come with writing that the Court has not denominated as an “opinion of the Court.” Consider, for instance, the unsigned order in Trump v. Wilcox, in which the Court provided 521 words of explanation to justify its stay of a pair of district court injunctions (for more on the ruling, see this post). In contrast, the “per curiam” opinion of the Court in Department of Education v. California (a case about the President’s cutting off of grants for teacher training to public schools with DEI-related initiatives) ran a grand total of 643 words. I don’t know why the Court internally chose to label the latter an “opinion of the Court,” but not the former.2 What does seem clear is that how the Court labels a ruling is not conclusive of whether or not it includes a written explanation. Thus, focusing on “opinions of the Court” is missing some cases with rationales (and, in the other direction, including some rulings in argued cases without rationales).
Second, and in any event, it seems increasingly difficult to justify excluding from data sets even unexplained rulings that have massive real-world impacts—like the unexplained grant of emergency relief in the third-country removal case, to take just the most recent example. It was one thing when the Court was handing down only a handful of these rulings each term. But in the last three months alone (since April 4), the Court has granted 17(!) applications for emergency relief. As these decisions are becoming an increasingly significant part of the Court’s overall workload, not counting them in our datasets seems increasingly difficult to justify.
To illustrate the difference, consider the following query: In how many cases during the October 2024 Term has the Court split 6-3, with the Democratic appointees all dissenting? If we look just at rulings that were argued and decided on the merits docket, the answer is 6. If we look at anything labeled an “opinion of the Court,” the answer (so far) is 7. But if we look at all opinions and emergency applications, the answer (so far) is 13. In other words, at the three-quarter-mark of the term, we’d still miss more than half of the most significant ideological divisions if we focused only on the merits docket; and just under half if we focused only on cases with “opinions of the Court.” It’s easy to downplay how often the justices are fracturing along ideological lines when we’re constructing the dataset in a way that excludes half of the cases in which that’s happening.3
Problem #3: The Circuit Scorecard
Finally, a common move in these end-of-term recaps is to focus on circuit-by-circuit scorecards. But here, again, we run into trouble depending upon what we do (or don’t do) with emergency applications. Picking on the SCOTUSblog StatPack again, that dataset appears to include grants of emergency relief in the circuit scorecard if they’re accompanied by an opinion of the Court, but not otherwise.
Separate from repeating the issues of counting only those emergency applications that are accompanied by opinions of the Court (exacerbated here by the fact that the few opinions the Court does write respecting emergency applications almost always come alongside grants of such relief), I’m also somewhat skeptical of including rulings on emergency applications as part of the circuit scorecard at all. For instance, SCOTUSblog describes the ruling in CASA, for instance, as “revers[ing] the U.S. Courts of Appeal [sic] for the First, Fourth, and Ninth Circuits.” But that’s not formally true. Granting a stay is not necessarily holding that the lower court was wrong; it’s holding only that the effect of the lower-court decision should be paused while the appeal proceeds. Indeed, we have numerous examples of the Court granting a stay and then ultimately affirming the decision below.4 Thus, the SCOTUSblog circuit scorecard includes a selective subset of emergency applications in a context in which I don’t think either the selectivity or the inclusion of any part of the subset makes sense. It may make sense to have a separate scorecard for emergency applications. But folding an arbitrary subset of them into the merits docket scorecard strikes me as a really difficult move to justify.
So far this term, at least, I don’t think that these hiccups make a huge difference; the Fifth Circuit still has the most total “losses,” and the First, Fourth, and Tenth Circuits all struck out. But given how much folks like to play up which courts of appeals had the best and worst scorecards, we ought to have more of a consensus about what is and isn’t included in the scorecard—and why.
Some of this may well seem like a critique of how we count the deck chairs on the Titanic. Fair enough. But especially at a moment when the Supreme Court is playing such an outsized role in so many areas of our contemporary discourse, it seems like we ought to be more careful about the claims that are informing that discourse. To that end, I have three very modest suggestions for how we should quantify end-of-term assessments going forward.
Solution #1: The Term Ends … When it Ends
This one really shouldn’t be hard. By statute, history, and tradition, the Court is defined by its annual terms, and its annual terms begin (and, since 1917, have begun) on the first Monday of October. That the justices have chosen to structure their argument sessions and opinion hand-downs so that their work is largely wrapped up by the end of June may be a strong norm, but it no longer has any formal significance whatsoever. In an age in which the decisions in argued cases on the merits docket comprised the vast majority of the newsworthy developments from the Supreme Court, having end-of-June/early-July wrap-up stories may have made more sense. But this term isn’t over in any meaningful way. And if Justice Kavanaugh has his way, the emergency docket is only going to get busier going forward.
Against that backdrop, it shouldn’t be that controversial to suggest that we measure the Court’s term … by reference to the actual term of the Court. Every ruling through the day before the first Monday in October should count—which, among other things, will also help to drive home the point that there’s plenty of action at the Court over the summer, even if it’s not as visible as what we’re used to in the spring. There’s nothing wrong, of course, with looking at where we are when the justices rise for their summer recess. But putting a bow on a term that is only 75% complete is going to lead to increasingly incomplete analyses.
Solution #2: Track All Full Court Rulings on Emergency Applications
In trying to figure out which emergency applications should also “count,” it doesn’t make sense for the answer to be “all of them.” The reality is that the overwhelming majority of applications are resolved by individual justices “in chambers,” and are thus not usually especially significant or divisive. For the last few years, I’ve tracked the results in only those applications that are referred to (and resolved by) the full Court—a subset that includes just about all of the ones we might care about (and some that we might not).
Looking at this term, as of this morning, there have been 108 full-Court rulings on emergency applications. If we pull out from that dataset those in which there was at least one public dissent, that produces a set of 34 decisions—a higher number than the total number of non-unanimous rulings on the merits docket (32). And within those 34 rulings, only one included dissenters from both ends of the ideological spectrum—the first ruling in the Dellinger emergency application, in which Justices Alito and Gorsuch would’ve granted the application in full and Justices Sotomayor and Jackson would’ve denied it in full (and the other five voted to hold the application in abeyance).
In other words, in a dataset involving more non-unanimous decisions so far this term than the merits docket, the public dissenters have almost uniformly come from one side of the ideological spectrum. That seems to make both a quantitative and qualitative case for tracking these kinds of rulings as their own category, going forward—no matter what form the disposition of the application takes.
Solution #3: Argued Merits Cases as Their Own Category
Finally, if we’re going to break out the Court’s decisions into more distinct categories, it seems to me that the traditional “merits” docket (that is to say, cases that are briefed, argued, and decided through written opinions after grants of certiorari) should be a category of its own—and one that is at least sometimes looked at separately from the full body of rulings by the full Court.
After all, “opinions of the Court” can come in numerous contexts in which the case has not received plenary review—including summary rulings at the certiorari stage or on emergency applications. But focusing on the cases that receive plenary review more accurately brings out the specific issues the justices chose to decide—and the (shrinking) total number of cases to which the Court agreed to direct its resources. Looking at this subset for this term, for instance, yields a total of 55 decisions (56 if we count the unsigned decision in TikTok)—which is tied for the second-lowest total since 1864. That remarkably low total gets obscured if we include every “opinion of the Court” (the Court’s website currently lists 67), even the ones that … don’t have any analysis.
***
Today’s bonus issue may have reached new heights (or depths) of Supreme Court nerdistry. But now is as good a time as any to point out that way too many folks who cover and talk about the Court approach it as if it were the Court of decades ago. As emergency applications become a more significant part of what the Court is actually doing, that’s exposing a bunch of flaws baked into the conventional modes of analyzing (and quantifying) Supreme Court terms. There are certainly more important things to be talking and worrying about these days, but if this piece has prompted folks to think at all about the strange and idiosyncratic ways that we “count” Supreme Court rulings, and how those approaches may distort public narratives about the Court’s overall output, then it will have been worth it.
We’ll be back (no later than) Monday with our next regular issue of “One First.” Until then, thank you for your continued support of this newsletter—and I hope you and yours are staying safe out there.
A “summary reversal,” in SCOTUS parlance, is not an unexplained reversal; rather, it is a reversal that comes at the certiorari stage—where, instead of granting certiorari and setting the case for plenary review, the justices write an opinion reversing the court of appeals and remanding the case without argument.
Alas, the possibility that the Court has a 600-word cutoff is belied by last August’s ruling in Department of Education v. California—in which the justices denied a pair of emergency applications from the Biden administration in a 541-word “per curiam” opinion of the Court.
Another common trope I’ve seen is that Justice Kagan was in the majority more often than Justices Thomas, Alito, and Gorsuch. Needless to say, that’s no longer true if we include non-unanimous rulings on emergency applications.
SCOTUSblog’s scorecard also does not count the three cases this term in which the Court issued a “DIG” (a dismissal of certiorari as improvidently granted) after oral argument, or the 4-4 ruling in the Oklahoma religious charter school case (which goes in the books as an “affirmance by an equally divided Court.” Leaving aside that all four of those rulings are labeled as “per curiam” opinions of the Court, I’ve always viewed such rulings as “wins” for the courts of appeals from which those cases originated, since the effect is to leave their decisions intact.
As usual, good analysis. Let's see if anyone at The Dispatch / SCOTUSblog reads your Substack.
I am just glad to see the stat pac is back. It is better than no stat pac and It dispels myths about the Court. Maybe Mr. Vladeck should use stat pac info as a base then add his own stats. StatPacPlus.