COVID scorecard

I’ve seen a lot of this country. I’ve been to 47 states. Every state has its own feel. Every state has its own local culture that makes it unique. This is even true of states that seem like they would be very much the same. Illinois is discernably different than Indiana, Missouri is not the same as Oklahoma, and so on. With that said, going from one state to another I’ve never really had the “out of place” feeling that I get when I travel internationally. At least for now, the COVID era has changed that. Every state is handling COVID in different ways. Many states look similar, but some states look wildly different from states that are right next door.

We went to visit family in Illinois over Thanksgiving. The kids hadn’t seen their grandparents in nearly a year. Tara’s mom went to a nursing home just before the pandemic hit. Since then her mom and her dad have been living extremely isolated existences. Many people were not able to see their extended family over Thanksgiving. We are thankful that we had the chance. Tara and I both have had COVID along with Tara’s brother, so we felt it was reasonably safe and well worth the risk. It was apparent as soon as we arrived that COVID in Illinois looks a lot different than COVID in Missouri. Every single restaurant is closed to indoor and outdoor dining. One of the few things to do in the town where they live is to go to Walmart. It was a very different experience. I saw only three people who were not wearing masks. Going in to the store, we passed a woman in PPE who was performing temperature checks. Many people were even abiding by the directional arrows on the floor of each aisle. The school district where my sister-in-law teaches has been having in-person classes all semester, but there are no sports or extracurriculars. As Missouri gets ready to crown a state champion in football next week, Illinois schools have yet to take a single snap. Basketball, far and away the most popular sport in the state, will likely be made into a spring sport if they get a season at all. I’ve crossed the bridge from Illinois to Missouri over 100 times. This was the first time where it felt a little like going from one country to another.

So, you have two states, both faced with the same pandemic, but handling it in very different ways. It invites the question, which state is right? Which states are handling COVID the best? It’s a tough question to answer for a couple of reasons. First, there isn’t universal agreement about what we mean by “best.” How do you judge success? Is it cases? Hospitalization? Perhaps it’s the most dubious of all metrics, test positivity rates? Deaths from COVID seems to be he best metric, but using that metric alone dismisses other collateral damage being done by governmental interventions. This leads to the second reason the question is difficult to answer. There is a bias towards doing as much as possible. Those states that are perceived to be doing something are naturally doing the best even if what they are doing has zero impact or even a negative impact. Imposing a curfew is seen as responsible even if it is pointless. This bias is clearly seen in the media. The media loves treating Florida and South Dakota as COVID punching bags because they are perceived to be doing nothing. Their governors are callous, uncaring, COVIDIOTS.

The problem is that the data doesn’t conveniently match the headlines.

What about South Dakota? The narrative is that they have done everything wrong. The governor’s stubborn refusal to adopt a masking mandate has led to unnecessary death and destruction. But has it? The reality is more complicated.

Causation is difficult to clearly identify when it comes to governmental policies. A governor could tell his citizens to hop on one foot for 3 minutes every day, and if this executive order goes into effect at the peak of the epidemic it will seem as if this executive order caused a drop in cases. If this executive order precedes a rise in cases, the governor can just claim that the citizens are not following the rules. That’s why a person can claim Missouri’s governor if they get sick. He just hasn’t done enough. But if someone gets sick in Illinois, the governor is absolved. It is the citizens who are to blame. That’s a nice little trick.

Because of our unique political system we actually have the ability to compare one state to another and identify what states are enacting policies that actually make a difference. If non-pharmaceutical interventions like curfews and closing restaurants made a big impact, you would expect the states who have implemented such policies to look much different from those who haven’t. Illinois should look much different than Missouri. The reality?

We should all want to know what states are handling COVID the best because it might help us to better prepare for the next crisis. It’s also the only way to hold our elected officials accountable for the decisions that they’ve made. Accountability is essential. Our officials shouldn’t be given the benefit of the doubt no matter their party affiliation. They deserve our skepticism. They are obligated to make their case to the people they govern. They have not done that. Too many have resorted to fear and shame rather than reason and evidence.

What I really want is some sort of COVID Scorecard. We need advanced analytics applied to each state six months after the pandemic to measure how effective they were. Here is what I would include in a state’s COVID score:

  1. Deaths from COVID/million residents. Six months after the pandemic we can get rid of all the confusing and misleading metrics like test positivity and just deal with deaths from COVID.
  2. Demographic distributions of death. We know that COVID effects older and at-risk populations significantly more than other populations. Many experts have argued that lockdowns protect the wrong people and redistribute suffering from the pandemic away from the middle and upper classes. Instead, lower and working class people – “essential” workers are left to shoulder a greater burden of the pandemic. This is already showing up in some data as experts like Martin Kulldorff are pointing out. In order to hold our leaders accountable, we need to discover if lockdowns have unjust consequences on certain communities of people. Conversely, in states that are less restrictive, is there a more equitable distribution of mortality?
  3. Excess deaths. This data is a little more controversial, but many statisticians have been tracking excess deaths during the pandemic. Most of these are not deaths directly from COVID. Many of these excess deaths are related to consequences of the lockdown like missed cancer screenings or so-called deaths of despair (suicides, over-doses, etc.).
  4. Percentage of businesses or organizations permanently closed by the pandemic. It’s hard to measure one state’s mitigation efforts against another’s. It’s easier to measure economic data which is the consequence of decisions like closing all indoor and outdoor dining. We should all want to know what long-term economic impact a state’s decisions have.
  5. Unemployment rate. Similar to business closures, a state’s unemployment rate is related to mitigation policies. A state’s unemployment rate six months after the pandemic is a really good indicator of the effect that mitigation efforts had on the economy.
  6. Days of school missed. This data is a little harder to find, but it is an important number. Keeping kids out of school will have a generational impact on those students. It is especially harmful to kids who were already at risk. This number must be included when assessing a state’s effectiveness.

There are perhaps other numbers that deserve inclusion in a COVID score. There are also some non-measurables that can’t be included. For instance, it it hard to measure misery and civil discord caused by arbitrary and ever-changing COVID restrictions. Nevertheless, a data-driven approach will actually reveal quite a bit. I know data like this would still be subject to interpretation. One person could say that COVID deaths outweigh every other data point. If we are given the choice between closing businesses and saving even one life, we should close as many businesses as necessary. Fine. But without having the actual numbers, such an argument is merely posturing. You could justify a lot of really bad and harmful policies if that is your only criteria. I also realize that something like this will likely not happen. There are too many people who stand to lose if data like this were put together and shared. If it does happen, I have the feeling that no matter what data is shared people would likely still find a way to explain it so that their biases are confirmed. Even in the midst of a pandemic, we seem to be less concerned about truth than we are about winning.

UPDATE: Someone has already taken it upon themselves to start gathering this data. They have combined COVID numbers from the CDC with a state coronavirus restriction rating to produce these charts. The conclusion is that there is no conclusion. There is no clear trend that shows that strict mitigation policies lead to less death or vice versa. It is left to us to wonder whether or not killing the economy and keeping kids out of school was really worth it.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s