Confirming the science behind COVID-19 policy recommendations

In early 2020 a group of behavioral scientists from around the world came up with about 20 science-based recommendations for public policymakers regarding COVID-19. Ellen Peters, the director of the University of Oregon’s Center for Science Communication Research, was among them. Those recommendations were published in the April 2020 issue of Nature and included messaging recommendations from social distancing to how to work with individual communities.

Now, nearly four years later, those same researchers — and an independent panel of scientists not involved in the original research — looked back at the real world data to see how accurate those recommendations were. Both groups found that about 84% of the original recommendations were accurate. Their findings have just been published in the December 2023 issue of Nature. We talk with Peters about some of the recommendations and what the results say about the scientific process and public’s confidence in science.

Note: The following transcript was created by a computer and edited by a volunteer.

Dave Miller: From the Gert Boyle Studio at OPB, this is Think Out Loud. I’m Dave Miller. In April of 2020, a group of behavioral researchers published a paper with 19 policy recommendations for how to lessen the effects of the then new COVID-19 pandemic. Many of those recommendations were taken up by governments and public health authorities all around the world. Now, the researchers behind that paper along with dozens of other scientists have, in a sense, checked their work. They analyzed more than 700 pandemic-related research articles to see if their recommendations actually worked. They found that the majority of them did, but the details are more interesting. Ellen Peters is one of the authors of both the 2020 paper and the new follow up. She is the director of the University of Oregon Center for Science Communication Research and she joins us once again. Welcome back to the show.

Ellen Peters: Hey, thanks. Nice to be here again.

Miller: It’s great to have you on again. What was the big idea of this analysis?

Peters: So we did this paper back in 2020. We, as best we could, based on all of our different sciences - because we all come from different aspects of behavioral science- were trying to come up with, “well, what do we think people should do in order to try to reduce the effects of the pandemic?” And so we did that, but we didn’t know if we were right because this is a brand new pandemic. And so our claims might have been wrong in the pandemic. They might have even backfired on us potentially. And so we just simply wanted to know, did behavioral science, and especially the behavioral science that we were bringing, offer anything during the pandemic?

Miller: How widespread was the uptake of your recommendations?

Peters: We know that different governments took it up, we know WHO talked about it. The lead author on that original paper, a guy named J. Van Bavel went to WHO and talked about the recommendations. So it was taken up pretty widely.

Miller: Broadly, what kinds of messages were most effective?

Peters: There were probably three or four things that were most effective. So if you’re talking about messaging in particular, messaging has to come from trusted leaders – so messages that come from someone that you trust at the CDC or a trusted church official, for example. Those are the ones that people are gonna be most likely to listen to and most likely to follow up on and do whatever was recommended. Other things, messages that emphasize what we call social norms, like things that other people are doing. So if I tell you, for example, that 78% of people got the COVID vaccine, meaning almost everybody got the COVID vaccine, then you’re gonna be more likely to actually get it. And that’s the idea of these positive social norms.

Miller: To me, the most surprising finding in the whole analysis is that messages that highlighted health benefits either for individuals - meaning if you do this thing, you’re more likely to not get sick, you’re more likely to be healthy - or messages that emphasize that if you do this thing, other people around you will be protected, those were not particularly effective. How do you explain that?

Peter: So I honestly am not quite sure how to explain it. These were predictions that we made because we thought that they would be followed through on. We thought that they would actually work or we wouldn’t have made the predictions. I think it’s the idea that a lot of these efforts that people took during the pandemic or didn’t take, they’re kind of costly at the individual level. And it takes effort to figure out what’s going on with the vaccines and go out of your way to get it and then get another one and then get another one, as we’ve needed to do. And that costliness, I think it was just too costly compared to the benefits that were coming out in those messages. So people just weighed it and some people decided the costs were greater.

Miller: But I’m confounded by it because in a sense, what this huge analysis of data from a lot of different countries found is that saying, “Everybody’s doing it,” is more effective than saying, “If you do this, you’ll be healthier.” And I guess we’re social beings and few of us want to be outliers and to not sort of do what everyone’s doing. But the message is, “Do this thing and you’re less likely to die.” I guess I don’t understand why many people wouldn’t heed that advice broadly?

Peters: Well, I think that that comes up against one of the other most strongly supported claims, which was the idea of combating misinformation. There was a lot of misinformation out there about vaccines, including about whether they actually were effective or not. The vaccines, of course, are effective. But I think the misinformation and the quantity of misinformation that was out there on social media among families, people [were] just giving information to one another that just simply wasn’t correct. It undermined the idea of the effectiveness.

Miller: There’s also the question of just how much we can know at the beginning of a public health emergency. So this April of 2020 paper emphasized the importance of hand washing. How relevant did that turn out to be?

Peters: Yeah. That’s a really great point. So when we were doing this paper we didn’t know very much - and science has grown and learned more as we’ve gone along - and at that point in time, we did think hand washing was very important. We were being told it was very important, but it ended up not being as important because as we know now, COVID-19 is a respiratory disease. And what that means is masking and ventilation are more important.

Miller: Right. And the early messaging that came from the CDC, people like Anthony Fauci, downplayed the use of masks early on, for example, and then obviously, later emphasized how helpful they were. That was the result of our evolving understanding of the risk of the virus. But that evolution, which uncharitably could be seen as a flip flop, was picked up by some segments of the public as a reason not to trust public health agencies.

How do you get around that? Because whatever the latest version in the future of a public health emergency is, it’s highly unlikely that the best scientists are gonna know everything at the beginning. So how can public health messaging let people understand that at the beginning, so they don’t mistrust those agencies later if and when messages change.

Peters: Yeah, I think we need to do a better job of educating people about the idea that science is really, “two steps forward and one step back.” But at any time we’re doing science, whoever the scientist is, is coming up with whatever they think the correct results are in that moment. But then we find out more later and it’s always kind of two steps forward, one step back. What we don’t do very well in communication during public health emergencies and even in other situations too - we don’t always communicate the notion that, “This is what we know right now but the situation might change.”

And what ends up happening is when the situation changes and in a pandemic, it changed constantly, people were surprised by it. They were disappointed by it. And people who wanted to do so, took advantage of it and really kind of sowed the seeds for mistrust.

Miller: Your analysis found no big difference between two phrases that were commonly used at the start of the pandemic: physical distancing and social distancing. Can you remind us what the thinking was more than three years ago about the potentially meaningful distinctions between these two phrases?

Peters: Well, Dave, you mentioned this notion that we’re social creatures and it was kind of based on that generic idea that if we talk about social distancing, well, who wants to be distanced from their social lives, who wants to be distanced from their loved ones, the people that they want to be social with? What we thought might be the case then is that if we could simply substitute the idea of physical distancing, which really is a better descriptor of what needed to happen – people just needed to physically be further apart while still being social - we thought that that “physical distancing” term would work better.

Miller: And what happened?

Peters: Nobody tested it. We actually don’t know. And so at the point that we did this analysis - again, we studied over 700 papers, I forget the exact number - there were no published papers that actually looked at the difference between the terms. And so we don’t actually know.

Miller: I mentioned Anthony Fauci earlier. I mentioned the CDC. It’s an American official and an American national health authority. I don’t know who the equivalent of Anthony Fauci is in Argentina or Romania or Zimbabwe, but this paper looked at studies from all around the world. Were there interesting differences in the way different messages were received and responded to globally?

Peters: The biggest thing that we concluded from this study is really the need to do more studies in countries outside of the United States and Western Europe – that we really needed to be doing more studies in Zimbabwe and in Africa, in South America.

Miller: The analysis found some missed opportunities in your early paper, things that could have been addressed in public health messaging that weren’t. What are some examples of that?

Peters: Some of the examples would be things like skepticism towards science and incentivizing behaviors beyond simply describing the benefits. Because remember before we talked about the idea that describing the benefits to the self in messages didn’t seem to make much of a difference. But instead, incentivizing behaviors could make a difference.

I should say though, just as a quick correction, Dave, it’s not so much that these things weren’t studied, it’s that we didn’t predict them. It’s that we missed these as predictions, but some of them are actually studied quite heavily. So “incentivizing behaviors,” for example, was studied quite a bit. Katy Milkman and some of her colleagues at the University of Pennsylvania and elsewhere did some amazing studies, partnering with, I think it was Walmart, where they provided financial rewards for vaccination and showed effect. And so it wasn’t so much that they weren’t studied, as we didn’t predict them and maybe we should have from our science.

Miller: If you were in charge of messaging for the next pandemic or next true health emergency at the federal level, what would your priorities be?

Peters: If I was in charge of messaging, I would end up emphasizing [and] finding trusted messengers, which isn’t always easy, because we can’t always use a single person from the CDC because that single person like Fauci, for example, was trusted by some people and not by others. And what that means is that greater efforts need to be put towards finding trusted messengers even down at that local level, finding trusted messengers within a church leader, within a very local area. And that’s just a lot more work, but it’s a really critical part. It is gonna be a really critical part of future kinds of health emergencies like pandemics.

I would also say we need to do interventions to combat misinformation. We need to emphasize those positive social norms that I was talking about. I’m trying to think what else I would suggest… Those are the biggest ones that I would key in on.

The other thing I would say though is we also do need more research on human behavior in these kinds of situations. I don’t know if you knew, but back in 2021, Francis Collins who was the NIH Director at the time, was just about to step down from being the director and he actually talked about the idea that maybe we under invested in research on human behavior. And our study is trying to get at the idea that there was investment in it. And we’re trying to take advantage of it here. So we made our predictions and then we went back to look to see if we’re right. But I do think we need more research on human behavior and that does require investment.

Miller: Ellen Peters, thanks very much.

Peters: Hey, thank you. It was a pleasure.

Miller: Ellen Peters is the director of the University of Oregon’s Center for Science Communication Research.

Contact “Think Out Loud®”

If you’d like to comment on any of the topics in this show or suggest a topic of your own, please get in touch with us on Facebook, send an email to thinkoutloud@opb.org, or you can leave a voicemail for us at 503-293-1983. The call-in phone number during the noon hour is 888-665-5865.