Vaccine experts say Moderna didn’t produce data critical to assessing Covid-19 vaccine

Heavy hearts soared Monday with news that Moderna’s Covid-19 vaccine candidate — the frontrunner in the American market — seemed to be generating an immune response in Phase 1 trial subjects. The company’s stock valuation also surged, hitting $29 billion, an astonishing feat for a company that currently sells zero products.

But was there good reason for so much enthusiasm? Several vaccine experts asked by STAT concluded that, based on the information made available by the Cambridge, Mass.-based company, there’s really no way to know how impressive — or not — the vaccine may be.

While Moderna blitzed the media, it revealed very little information — and most of what it did disclose were words, not data. That’s important: If you ask scientists to read a journal article, they will scour data tables, not corporate statements. With science, numbers speak much louder than words.

Even the figures the company did release don’t mean much on their own, because critical information — effectively the key to interpreting them — was withheld.

Experts suggest we ought to take the early readout with a big grain of salt. Here are a few reasons why.

The silence of the NIAID

The National Institute for Allergy and Infectious Diseases has partnered with Moderna on this vaccine. Scientists at NIAID made the vaccine’s construct, or prototype, and the agency is running the Phase 1 trial. This week’s Moderna readout came from the earliest of data from the NIAID-led Phase 1.

NIAID doesn’t hide its light under a bushel. The institute generally trumpets its findings, often offering director Anthony Fauci — who, fair enough, is pretty busy these days — or other senior personnel for interviews.

But NIAID did not put out a press release Monday and declined to provide comment on Moderna’s announcement.

The n = 8 thing

The company’s statement led with the fact that all 45 subjects (in this analysis) who received doses of 25 micrograms (two doses each), 100 micrograms (two doses each), or a 250 micrograms (one dose) developed binding antibodies.

Later, the statement indicated that eight volunteers — four each from the 25-microgram and 100-microgram arms — developed neutralizing antibodies. Of the two types, these are the ones you’d really want to see.

We don’t know results from the other 37 trial participants. This doesn’t mean that they didn’t develop neutralizing antibodies. Testing for neutralizing antibodies is more time-consuming than other antibody tests and must be done in a biosecurity level 3 laboratory. Moderna disclosed the findings from eight subjects because that’s all it had at that point. Still, it’s a reason for caution.

Separately, while the Phase 1 trial included healthy volunteers ages 18 to 55 years, the exact ages of these eight people are unknown. If, by chance, they mostly clustered around the younger end of the age spectrum, you might expect a better response to the vaccine than if they were mostly from the senior end of it. And given who is at highest risk from the SARS-CoV-2 coronavirus, protecting older adults is what Covid-19 vaccines need to do.

There’s no way to know how durable the response will be

The report of neutralizing antibodies in subjects who were vaccinated comes from blood drawn two weeks after they received their second dose of vaccine.

Two weeks.

“That’s very early. We don’t know if those antibodies are durable,” said Anna Durbin, a vaccine researcher at Johns Hopkins University.

There’s no real way to contextualize the findings

Moderna stated that the antibody levels seen were on a par with — or greater than, in the case of the 100-microgram dose — those seen in people who have recovered from Covid-19 infection.

But studies have shown antibody levels among people who have recovered from the illness vary enormously; the range that may be influenced by the severity of a person’s disease. John “Jack” Rose, a vaccine researcher from Yale University, pointed STAT to a study from China that showed that, among 175 recovered Covid-19 patients studied, 10 had no detectable neutralizing antibodies. Recovered patients at the other end of the spectrum had really high antibody levels.

So though the company said the antibody levels induced by vaccine were as good as those generated by infection, there’s no real way to know what that comparison means.

STAT asked Moderna for information on the antibody levels it used as a comparator. The response: That will be disclosed in an eventual journal article from NIAID, which is part of the National Institutes of Health.

“The convalescent sera levels are not being detailed in our data readout, but would be expected in a downstream full data exposition with NIH and its academic collaborators,” Colleen Hussey, the company’s senior manager for corporate communications, said in an email.

Durbin was struck by the wording of the company’s statement, pointing to this sentence: “The levels of neutralizing antibodies at day 43 were at or above levels generally seen in convalescent sera.”

“I thought: Generally? What does that mean?” Durbin said. Her question, for the time being, can’t be answered.

Rose said the company should disclose the information. “When a company like Moderna with such incredibly vast resources says they have generated SARS-2 neutralizing antibodies in a human trial, I would really like to see numbers from whatever assay they are using,” he said.

Moderna’s approach to disclosure

The company has not yet brought a vaccine to market, but it has a variety of vaccines for infectious diseases in its pipeline. It doesn’t publish on its work in scientific journals. What is known has been disclosed through press releases. That’s not enough to generate confidence within the scientific community.

“My guess is that their numbers are marginal or they would say more,” Rose said about the company’s SARS-2 vaccine, echoing a suspicion that others have about some of the company’s other work.

“I do think it’s a bit of a concern that they haven’t published the results of any of their ongoing trials that they mention in their press release. They have not published any of that,” Durbin noted.

Still, she characterized herself as “cautiously optimistic” based on what the company has said so far.

“I would like to see the data to make my own interpretation of the data. But I think it is at least encouraging that we’ve seen immune responses with this RNA vaccine that we haven’t seen with previous RNA vaccines for other pathogens. Whether it’s going to be enough, we don’t know,” Durbin said.

Moderna has been more forthcoming with data on at least one of its other vaccine candidates. In a statement issued in January about a Phase 1 trial for its cytomegalovirus (CMV) vaccine, it quantified how far over baseline measures antibody levels rose in vaccines.