In a recent study published in the journal Science, researchers investigated the impact of Facebook misinformation and vaccine-skeptical content on coronavirus disease 2019 (COVID-19) vaccination intentions in the United States.

Their results indicate that while flagged misinformation reduces vaccination intentions, the far greater exposure to unflagged vaccine-skeptical content from mainstream sources significantly impacted vaccine hesitancy.

Study: Quantifying the impact of misinformation and vaccine-skeptical content on Facebook. Image Credit: Accogliente Design / Shutterstock

Background

In recent years, online misinformation has garnered significant attention due to its perceived real-world harm, as evidenced by events like public health message rejection during the COVID-19 pandemic.

Despite extensive research on the spread and psychology of misinformation, its direct causal impact on behaviors such as vaccine uptake remains underexplored.

This gap is particularly crucial regarding COVID-19 vaccine misinformation, where correlations have been found between misinformation exposure and lower vaccination rates, but the direction and causality of this relationship are unclear.

Additionally, content that is not factually inaccurate but raises vaccine skepticism could contribute significantly to vaccine hesitancy.

About the study

Addressing existing knowledge gaps, this study introduced a framework combining experimental data and Facebook exposure data to quantify the causal impact of flagged misinformation and unflagged vaccine-skeptical content on COVID-19 vaccination intentions.

The study explored these effects using two large-scale survey experiments conducted on an online platform involving 18,725 American participants.

The first experiment included 8,603 participants, each exposed to either a neutral control post or one of 40 pieces of vaccine misinformation that fact-checkers had previously debunked.

The impact on vaccination intentions was assessed before and after exposure, with responses combined into a COVID-19 vaccination index. The effect of exposure on vaccination intentions was analyzed using statistical methods, and the significance of the results was determined using the Benjamini-Hochberg correction.

In the second experiment, researchers showed 10,122 participants 90 highly shared vaccine-related articles from Facebook, representing various topics and domain qualities.

Pre- and post-exposure methods used to measure vaccination intentions were similar to those used in the first experiment. The causal effects of each piece of content on vaccination intentions were assessed. A random-effects meta-regression was conducted using five headline-level features (surprising, plausible, favorable to Democrats vs. Republicans, familiar, harmful vs. helpful to health).

To examine exposure to vaccine-related content on Facebook, the Social Science One dataset, which included 13,206 links about the COVID-19 vaccine shared publicly at least 100 times on Facebook during the first three months of 2021, was used.

Crowdsourcing and machine learning were employed to generalize the survey results to this dataset. Crowd raters were used to predict the effect of the headlines on vaccination intentions. These predictions were aggregated and used in a random-effects meta-regression to estimate the treatment effects for the full set of links.

Findings

Flagged misinformation received 8.7 million views on Facebook, accounting for 0.3% of all vaccine-related views. While flagged misinformation had a significant negative impact, it was overshadowed by the more widespread vaccine-skeptical content not flagged as misinformation.

Machine learning predictions used to generate scaled impact estimates suggested that vaccine-skeptical content not flagged by fact-checkers had a much more significant overall negative effect than flagged misinformation.

The first experiment found that exposure to a single piece of vaccine misinformation decreased vaccination intentions by an average of 1.5 percentage points.

This effect varied significantly across different pieces of misinformation, with the most impactful misinformation reducing intentions by 3 percentage points and others having no effect.

The second experiment found that the extent to which headlines suggested that the vaccine was harmful to health was the only consistent predictor of decreased vaccination intentions. Overall, false claims had a more negative effect on vaccination intentions, but this was primarily because they were more likely to suggest harm.

Specifically, a one-point increase in harmfulness perception was associated with a reduction of 0.69 percentage points in vaccination intentions when this was the only predictor and 0.49 percentage points when other predictors were included.

Conclusions

Overall, the study highlights the substantial impact of vaccine-skeptical content, especially from mainstream sources, on reducing vaccination intentions compared to flagged misinformation.

The study introduced a method integrating crowdsourcing, machine learning, and large-scale data to estimate social media’s causal effects on societal outcomes, specifically vaccine hesitancy.

The findings suggest that unflagged, vaccine-skeptical content from mainstream media significantly reduced vaccination intentions by 2.3 percentage points per Facebook user. This indicates that current misinformation interventions are inadequate for such content.

The study’s strengths include its novel approach and robust checks, while limitations involve timing discrepancies between data sources and the focus on intentions over actual behavior.

Future research should explore the impact of non-link content and translate survey results into social media contexts. The approach offers a replicable framework for assessing harmful content impacts.