As the COVID-19 pandemic took off during early 2020, widespread interest in modeling the trajectory of infections emerged. This interest was predicated on the hope that accurate models could be developed and subsequently used to help governments and policy makers monitor the effect of lockdowns and determine safe points in time to reopen.

Directly observing risky interactions (where transmission could occur) was not possible at scale, so researchers considered measuring movement trends across workplaces, transit systems, public parks, grocery stores, and pharmacies. This mobility data was obtained from aggregated, anonymized cell phone GPS data and lauded as a promising proxy for risky person-to-person interactions.

Indeed, initial research suggested strong associations between mobility and infection rates. However, it was unclear whether these observed associations would remain stable over the course of the pandemic as personal hygiene, mask wearing, and social distancing evolved. Our recent work accepted in Nature Digital Medicine focused on this question.

In this post, we examine some of our findings. We illustrate the temporal and geographic variability present in infection rate and mobility data, and develop a flexible, interpretable class of models to assess how the effect of mobility changes. Using one year of data across 94% of all 3,142 US counties (corresponding to 99.7% of the US population), we show that during the initial phases of the pandemic, strong association between mobility and infection rate is not robust outside of large metropolitan areas. We further show that our findings are sensitive to model flexibility. We conclude that a complete understanding of this relationship is difficult to pin down due to data challenges arising from coarse proxy variables for the volume of transmission events.

County-level infection incidence and mobility over the course of the pandemic

We quantified the spread of COVID-19 through weekly log infection growth rates, where the underlying infection incidence was estimated from county-level case counts. Figure 1 shows county-level weekly infection growth rates from February 2020 to February 2021. Broadly speaking, counties in the same combined statistical area (CSA) often experienced outbreaks together, and outbreaks in the same US Census Division were roughly synchronized. However, the overall infection growth rates were very heterogeneous across geographies and time.

We used Google’s mobility trends as an aggregate measure of mobility across six categories: grocery/pharmacy, residential, retail/recreation, workplace, transit, and parks. As provided, these data are normalized using pre-pandemic data to adjust for differences in population and mobility levels across counties. The data show that mobility rapidly declined at the start of the pandemic due to widespread lockdowns and changes in behavior. We also observed, in the data, clustering across CSAs and divisions.

Care is needed for meaningful inference

We proposed a multilevel regression framework where the expected county-by-county, weekly log infection growth rate is estimated by a linear function of mobility, temperature, and population, and by state-level mask use. To address the observed spatiotemporal clustering in infection rates and mobility, we allowed the effect of mobility to vary over time, and constrained this effect to be constant over geographies.

A key finding of our work was that model conclusions were sensitive to time-varying and geographic constraints. In particular, overly flexible models lead to misleading inferences, while overly rigid models obscure important effects. Figure 2 illustrates potential pitfalls for three overly flexible variations on our base model.

We also considered a set of simpler models in which mobility was allowed to vary geographically across CSA (region), or was fixed nationally and allowed to vary temporally across four waves, or was fixed across time. We found that coarser spatial and temporal constraints obscured effects and could not be used to identify meaningful relationships at the county level.

As a result, our final model constrained a combined mobility metric (defined as the first principle component of Google’s six mobility metrics) to vary temporally as a piece-wise constant function across four 13-week waves of the pandemic (February-May 2020; May-August 2020; August-October 2020; October 2020-February 2021). We also allowed the effect of mobility to vary geographically across CSAs.

These finer constraints led to interpretable models with meaningful conclusions. Figure 3 illustrates the combined mobility metric, estimated effects of mobility, and model fits and observed infection rates for three CSAs. We see that the effect of mobility varies over time and can differ starkly over geographies; the effect of mobility in New York was high during the first two waves, whereas in San Jose and Green Bay mobility had zero effect on infection rates during those periods. In Green Bay, in the second to fourth waves, mobility had a large effect on infection rate.

Mobility is most predictive in larger metro areas during spring 2020

Next, we examine the aggregate performance of our model. To measure our aggregate mobility model’s effectiveness at predicting infection growth rate, we used $R^2$. This commonly used statistical measure roughly indicates the proportion of inherent variability of the infection growth rate that the model can explain. Higher $R^2$ values correspond to better infection growth-rate predictions. To assess model performance we computed $R^2$ across various time periods and geographies.

Figure 4 shows that the overall $R^2$ is highest during spring 2020, and especially for the largest counties; this finding is consistent with prior work. However, $R^2$ is much lower for the roughly 45% of counties that have a population less than 25,000 people. The $R^2$ decreases in the fall and early winter of 2020 for counties of all sizes, and slightly elevates in the period from December 2020 to January 2021.

Figure 5 illustrates the estimated effect of mobility for each county alongside the model $R^2$. Estimated effects are highly non-stationary across time and geography; consider the change in effect in the South, West, and Midwest over the course of the four waves. Even with strong estimated effects, the predictive ability of mobility can remain low. This is particularly striking in the fourth wave, from November 2020 to February 2021.

Conclusion

A complete understanding of the relationship between mobility and infection rates is difficult to achieve. We showed that aggregate mobility metrics are only a coarse predictor for possible risky infection transmission events. The quality and reliability of this proxy variable is subject to several confounding factors, such as individual and group behavior, that we expect evolved over the course of the pandemic.

Our mobility model was predictive in the first wave of the pandemic and reproduced prior results. However, our extended analysis across the first year of the pandemic, with models that adjust for time varying effects, demonstrated that cell phone mobility is not a reliable leading indicator for the COVID-19 infection rate.

There are several opportunities to extend our work.

From a data quality perspective, we did not attempt to adjust for geographic and temporal non-stationary testing artifacts—such as differing reporting standards or availability of tests across different states. Moreover, the time-varying mask variable had relatively poor coverage, especially at the start of the pandemic, a period during which we would ideally tease apart the effect of masks from large drops in mobility. Finally, our analysis covered a period before widespread vaccine availability and the emergence of more infectious variants of the virus.

From a model perspective, ideally we would identify and estimate a mechanistic model (such as a compartmental model) that explicitly models transitions from susceptible, infected, and recovered subgroups of the population. However, data limitations make such models difficult to identify and estimate, so we restricted our approach to statistical models that captured associations between mobility and infection growth rates.

In the time since we began our work in late summer 2020, the COVID-19 landscape has drastically changed. Back then, vaccine efficacy and potential inoculation timelines were uncertain. We knew that respiratory virus transmission is more prevalent in the colder weather, and that holiday gatherings lead to outbreaks. As a result, researchers in many countries sought data and models to provide early warnings of emerging outbreaks or to justify partial reopening strategies.

Fast-forwarding to February 2022, we have widespread availability of extremely effective vaccines, along with emergence of new variants with unknown susceptibility to existing interventions. This new landscape, combined with our findings that the relationship between mobility and infection transmission is unreliable, suggests that mobility in isolation is less important than first thought. Nonetheless, we hope that our modeling explorations, and the lessons learned from studying the evolving COVID-19 landscape, are interesting and useful to modelers and practitioners going forward.

Acknowledgements

Many people contributed to this work including Sean Jewell, Joe Futoma, Lauren Hannah, Andrew C. Miller, Nicholas J. Foti, and Emily B. Fox.

Resources

Read the paper and view source code “It’s Complicated: Characterizing The Time-varying Relationship Between Cell Phone Mobility and COVID-19 Spread in the US” which was accepted at Nature Digital Medicine.

Read the paper and article published on Nature Digital Medicine.

References

Badr, Hamada S., Hongru Du, Maximilian Marshall, Ensheng Dong, Marietta M. Squire, and Lauren M. Gardner. "Association between mobility patterns and COVID-19 transmission in the USA: a mathematical modelling study." The Lancet Infectious Diseases 20, no. 11 (2020): 1247-1254. link

Chang, Serina, Emma Pierson, Pang Wei Koh, Jaline Gerardin, Beth Redbird, David Grusky, and Jure Leskovec. "Mobility network models of COVID-19 explain inequities and inform reopening." Nature 589, no. 7840 (2021): 82-87. link


IHME COVID-19 forecasting team. "Modeling COVID-19 scenarios for the United States." Nature Medicine (2020). link

Jewell, Sean, Joseph Futoma, Lauren Hannah, Andrew C. Miller, Nicholas Foti, Emily B. Fox. “It’s complicated: characterizing the time-varying relationship between cell phone mobility and COVID-19 spread in the US”. Nature Digital Medicine 4, no. 1 (2021). link

Unwin, H. Juliette T., Swapnil Mishra, Valerie C. Bradley, Axel Gandy, Thomas A. Mellan, Helen Coupland, Jonathan Ish-Horowicz et al. "State-level tracking of COVID-19 in the United States." Nature communications 11, no. 1 (2020): 1-9. link


Xiong, Chenfeng, Songhua Hu, Mofeng Yang, Weiyu Luo, and Lei Zhang. "Mobile device data reveal the dynamics in a positive relationship between human mobility and COVID-19 infections." Proceedings of the National Academy of Sciences 117, no. 44 (2020): 27087-27089. link


Zhou, Yiwang, Lili Wang, Leyao Zhang, Lan Shi, Kangping Yang, Jie He, Bangyao Zhao, William Overton, Soumik Purkayastha, and Peter Song. "A spatiotemporal epidemiological prediction model to inform county-level COVID-19 risk in the United States." Harvard Data Science Review (2020). link