Students Use AI for a Better COVID-19 Prediction Model

A team of Caltech students, led by Yaser Abu-Mostafa (PhD '83), have developed a tool to predict the impact of COVID-19 using artificial intelligence (AI).

The project, began as an assignment for Abu-Mostafa's computer science class, CS/CNS/EE 156 on Learning Systems, during Caltech's spring term. It has blossomed into a race to create a tool for policymakers to assess the pandemic's potential impact on their community and to predict the effect of mitigation efforts.

The idea to redirect the focus of CS/CNS/EE 156 came from Caltech senior Alexander Zlokapa, who runs the Caltech Data Science Organization, a student initiative to promote student engagement with data science research. Zlokapa approached Abu-Mostafa on March 19 to suggest switching gears from the course's original topic: the use of AI to make movie recommendations.

Worried about changing gears at the last minute, especially from a fun, light topic to something a lot more serious, Abu-Mostafa emailed his students for their thoughts. “The response was overwhelming. We all saw an opportunity to make a real difference for the country,” says Abu-Mostafa, professor of electrical engineering and computer science. Eighty students were already in the class; an additional 70 signed up when the new challenge was announced.

“Even at that early stage, the gravity of the pandemic was becoming clear: we are at war, and the enemy is potent,” Abu-Mostafa says. “We do not yet have the weapons to fight it, neither a cure nor a vaccine. However, there is another essential weapon in war; intelligence. We need to know what the enemy is doing so we can plan ahead. My students and I can help gather this intelligence using, fittingly, artificial intelligence, the subject of my course.”

“The standard epidemiology models make basic assumptions about the way a disease will spread, and then let you make tweaks based on infection and recovery rates and so on. AI, on the other hand, takes nothing for granted,” says graduate student Dominic Yurk (BS '17), who was the head teaching assistant for the course.

If you are attempting to predict the future spread of COVID-19 in a given area, you do not want to look at what happened before and just draw a line going forward. You want to take into account population density and mobility, whether people are going to restaurants and to work, the hospital capacity in the area, the incidence of flu in the past; there is a lot of data that can be relevant to this problem, according to the researchers.

The class's more than 150 students, who worked in competitive teams of up to four through April and May, were tasked with making accurate predictions about the impact of the disease. They focused on predicting mortality rates, which were the most reliable metric for assessing the impact of the disease in a given location, given that testing data tends to be unreliable and not necessarily comprehensive. They spent thousands of hours on the project. They gathered any data that seemed relevant, including COVID-19 mortality data, population demographics and densities of affected communities, whether people are abiding by stay-at-home orders, clinical statistics, previous flu data, and so on.

“We set the rules of the competition with policymakers in mind,” Abu-Mostafa says. “Here is an example of what to avoid: At some point in April, the state of Georgia had 590 intensive care unit (ICU) beds available. At that time, a prominent model predicted that Georgia's COVID-19 patients would need between 424 and 1,928 ICU beds at any given time. Such an imprecise prediction is of no help to policymakers,” Abu-Mostafa says.

In the end, about 40 teams produced viable models capable of making reasonable predictions, and the top 10 or so were competitive with existing models in terms of their ability to make precise predictions about the future impact of COVID-19 on a given community. 

When the class concluded in June, Abu-Mostafa's students told him they wanted to continue the effort over the summer—to take the best parts of the top models generated by the class and aggregate them into a single more powerful tool. He selected eight graduate students (including Yurk) and 18 undergraduate students and tasked his new team with creating a model with an expanded scope, making rapid and precise predictions about mortality rates, number of infections, and positivity rates of testing.

Sources: Press materials received from the company and additional information gleaned from the company’s website.