[quote][pid=484504334,25071596,1]Reply[/pid] Post by [uid=42361762]4862555c[/uid] (2021-01-12 04:21):
你先把链接发一下[/quote]Peter Doshi: Pfizer and Moderna’s “95% effective” vaccines—let’s be cautious and first see the full data
November 26, 2020
Only full transparency and rigorous scrutiny of the data will allow for informed decision making, argues Peter Doshi
In the United States, all eyes are on Pfizer and Moderna. The topline efficacy results from their experimental covid-19 vaccine trials are astounding at first glance. Pfizer says it recorded 170 covid-19 cases (in 44,000 volunteers), with a remarkable split: 162 in the placebo group versus 8 in the vaccine group. Meanwhile Moderna says 95 of 30,000 volunteers in its ongoing trial got covid-19: 90 on placebo versus 5 receiving the vaccine, leading both companies to claim around 95% efficacy.
Let’s put this in perspective. First, a relative risk reduction is being reported, not absolute risk reduction, which appears to be less than 1%. Second, these results refer to the trials’ primary endpoint of covid-19 of essentially any severity, and importantly not the vaccine’s ability to save lives, nor the ability to prevent infection, nor the efficacy in important subgroups (e.g. frail elderly). Those still remain unknown. Third, these results reflect a time point relatively soon after vaccination, and we know nothing about vaccine performance at 3, 6, or 12 months, so cannot compare these efficacy numbers against other vaccines like influenza vaccines (which are judged over a season). Fourth, children, adolescents, and immunocompromised individuals were largely excluded from the trials, so we still lack any data on these important populations.
I previously argued that the trials are studying the wrong endpoint, and for an urgent need to correct course and study more important endpoints like prevention of severe disease and transmission in high risk people. Yet, despite the existence of regulatory mechanisms for ensuring vaccine access while keeping the authorization bar high (which would allow placebo-controlled trials to continue long enough to answer the important question), it’s hard to avoid the impression that sponsors are claiming victory and wrapping up their trials (Pfizer has already sent trial participants a letter discussing “crossing over” from placebo to vaccine), and the FDA will now be under enormous pressure to rapidly authorize the vaccines.
But as conversation shifts to vaccine distribution, let’s not lose sight of the evidence. Independent scrutiny of the underlying trial data will increase trust and credibility of the results. There also might be important limitations to the trial findings we need to be aware of.
Most crucially, we need data-driven assurances that the studies were not inadvertently unblinded, by which I mean investigators or volunteers could make reasonable guesses as to which group they were in. Blinding is most important when measuring subjective endpoints like symptomatic covid-19, and differences in post-injection side-effects between vaccine and placebo might have allowed for educated guessing. Past placebo-controlled trials of influenza vaccine were not able to fully maintain blinding of vaccine status, and the recent “half dose” mishap in the Oxford covid-19 vaccine trial was apparently only noticed because of milder-than-expected side-effects. (And that is just one of many concerns with the Oxford trial.)
In contrast to a normal saline placebo, early phase trials suggested that systemic and local adverse events are common in those receiving vaccine. In one Pfizer trial, for example, more than half of the vaccinated participants experienced headache, muscle pain and chills—but the early phase trials were small, with large margins of error around the data. Few details from the large phase 3 studies have been released thus far. Moderna’s press release states that 9% experienced grade 3 myalgia and 10% grade 3 fatigue; Pfizer’s statement reported 3.8% experienced grade 3 fatigue and 2% grade 3 headache. Grade 3 adverse events are considered severe, defined as preventing daily activity. Mild and moderate severity reactions are bound to be far more common.
One way the trial’s raw data could facilitate an informed judgment as to whether any potential unblinding might have affected the results is by analyzing how often people with symptoms of covid-19 were referred for confirmatory SARS-CoV-2 testing. Without a referral for testing, a suspected covid-19 case could not become a confirmed covid-19 case, and thus is a crucial step in order to be counted as a primary event: lab-confirmed, symptomatic covid-19. Because some of the adverse reactions to the vaccine are themselves also symptoms of covid-19 (e.g. fever, muscle pain), one might expect a far larger proportion of people receiving vaccine to have been swabbed and tested for SARS-CoV-2 than those receiving placebo.
This assumes all people with symptoms would be tested, as one might expect would be the case. However the trial protocols for Moderna and Pfizer’s studies contain explicit language instructing investigators to use their clinical judgment to decide whether to refer people for testing. Moderna puts it this way:
“It is important to note that some of the symptoms of COVID-19 overlap with solicited systemic ARs that are expected after vaccination with mRNA-1273 (eg, myalgia, headache, fever, and chills). During the first 7 days after vaccination, when these solicited ARs are common, Investigators should use their clinical judgement to decide if an NP swab should be collected.”
This amounts to asking investigators to make guesses as to which intervention group patients were in. But when the disease and the vaccine side-effects overlap, how is a clinician to judge the cause without a test? And why were they asked, anyway?
Importantly, the instructions only refer to the first seven days following vaccination, leaving unclear what role clinician judgment could play in the key days afterward, when cases of covid-19 could begin counting towards the primary endpoint. (For Pfizer, 7 days after the 2nd dose. For Moderna, 14 days.)
In a proper trial, all cases of covid-19 should have been recorded, no matter which arm of the trial the case occurred in. (In epidemiology terms, there should be no ascertainment bias, or differential measurement error). It’s even become common sense in the Covid era: “test, test, test.” But if referrals for testing were not provided to all individuals with symptoms of covid-19—for example because an assumption was made that the symptoms were due to side-effects of the vaccine—cases could go uncounted.
Data on pain and fever reducing medicines also deserve scrutiny. Symptoms resulting from a SARS-CoV-2 infection (e.g. fever or body aches) can be suppressed by pain and fever reducing medicines. If people in the vaccine arm took such medicines prophylactically, more often, or for a longer duration of time than those in the placebo arm, this could have led to greater suppression of covid-19 symptoms following SARS-CoV-2 infection in the vaccine arm, translating into a reduced likelihood of being suspected for covid-19, reduced likelihood of testing, and therefore reduced likelihood of meeting the primary endpoint. But in such a scenario, the effect was driven by the medicines, not the vaccine.
Neither Moderna nor Pfizer have released any samples of written materials provided to patients, so it is unclear what, if any, instructions patients were given regarding the use of medicines to treat side effects following vaccination, but the informed consent form for Johnson and Johnson’s vaccine trial provides such a recommendation:
“Following administration of Ad26.COV2.S, fever, muscle aches and headache appear to be more common in younger adults and can be severe. For this reason, we recommend you take a fever reducer or pain reliever if symptoms appear after receiving the vaccination, or upon your study doctor’s recommendation.”
There may be much more complexity to the “95% effective” announcement than meets the eye—or perhaps not. Only full transparency and rigorous scrutiny of the data will allow for informed decision making. The data must be made public.
Five weeks ago, when I raised questions about the results of Pfizer’s and Moderna’s covid-19 vaccine trials, all that was in the public domain were the study protocols and a few press releases. Today, two journal publications and around 400 pages of summary data are available in the form of multiple reports presented by and to the FDA prior to the agency’s emergency authorization of each company’s mRNA vaccine. While some of the additional details are reassuring, some are not. Here I outline new concerns about the trustworthiness and meaningfulness of the reported efficacy results.
“Suspected covid-19”
All attention has focused on the dramatic efficacy results: Pfizer reported 170 PCR confirmed covid-19 cases, split 8 to 162 between vaccine and placebo groups. But these numbers were dwarfed by a category of disease called “suspected covid-19”—those with symptomatic covid-19 that were not PCR confirmed. According to FDA’s report on Pfizer’s vaccine, there were “3410 total cases of suspected, but unconfirmed covid-19 in the overall study population, 1594 occurred in the vaccine group vs. 1816 in the placebo group.”
With 20 times more suspected than confirmed cases, this category of disease cannot be ignored simply because there was no positive PCR test result. Indeed this makes it all the more urgent to understand. A rough estimate of vaccine efficacy against developing covid-19 symptoms, with or without a positive PCR test result, would be a relative risk reduction of 19% (see footnote)—far below the 50% effectiveness threshold for authorization set by regulators. Even after removing cases occurring within 7 days of vaccination (409 on Pfizer’s vaccine vs. 287 on placebo), which should include the majority of symptoms due to short-term vaccine reactogenicity, vaccine efficacy remains low: 29% (see footnote).
If many or most of these suspected cases were in people who had a false negative PCR test result, this would dramatically decrease vaccine efficacy. But considering that influenza-like illnesses have always had myriad causes—rhinoviruses, influenza viruses, other coronaviruses, adenoviruses, respiratory syncytial virus, etc.—some or many of the suspected covid-19 cases may be due to a different causative agent.
But why should etiology matter? If those experiencing “suspected covid-19” had essentially the same clinical course as confirmed covid-19, then “suspected plus confirmed covid-19” may be a more clinically meaningful endpoint than just confirmed covid-19.
However, if confirmed covid-19 is on average more severe than suspected covid-19, we must still keep in mind that at the end of the day, it is not average clinical severity that matters, it’s the incidence of severe disease that affects hospital admissions. With 20 times more suspected covid-19 than confirmed covid-19, and trials not designed to assess whether the vaccines can interrupt viral transmission, an analysis of severe disease irrespective of etiologic agent—namely, rates of hospitalizations, ICU cases, and deaths amongst trial participants—seems warranted, and is the only way to assess the vaccines’ real ability to take the edge off the pandemic.
There is a clear need for data to answer these questions, but Pfizer’s 92-page report didn’t mention the 3410 “suspected covid-19” cases. Nor did its publication in the New England Journal of Medicine. Nor did any of the reports on Moderna’s vaccine. The only source that appears to have reported it is FDA’s review of Pfizer’s vaccine.
The 371 individuals excluded from Pfizer vaccine efficacy analysis
Another reason we need more data is to analyse an unexplained detail found in a table of FDA’s review of Pfizer’s vaccine: 371 individuals excluded from the efficacy analysis for “important protocol deviations on or prior to 7 days after Dose 2.” What is concerning is the imbalance between randomized groups in the number of these excluded individuals: 311 from the vaccine group vs 60 on placebo. (In contrast, in Moderna’s trial, there were just 36 participants excluded from the efficacy analysis for “major protocol deviation”—12 vaccine group vs 24 placebo group.)
What were these protocol deviations in Pfizer’s study, and why were there five times more participants excluded in the vaccine group? The FDA report doesn’t say, and these exclusions are difficult to even spot in Pfizer’s report and journal publication.
Fever and pain medications, unblinding, and primary event adjudication committees
Last month I expressed concern about the potential confounding role of pain and fever medications to treat symptoms. I posited that such drugs could mask symptoms, leading to underdetection of covid-19 cases, possibly in greater numbers in people who received the vaccine in an effort to prevent or treat adverse events. However, it seems their potential to confound results was fairly limited: although the results indicate that these medicines were taken around 34 times more often in vaccine versus placebo recipients (at least for Pfizer’s vaccine—Moderna did not report as clearly), their use was presumably concentrated in the first week after vaccine use, taken to relieve post-injection local and systemic adverse events. But the cumulative incidence curves suggest a fairly constant rate of confirmed covid-19 cases over time, with symptom onset dates extending well beyond a week after dosing.
That said, the higher rate of medication use in the vaccine arm provides further reason to worry about unofficial unblinding. Given the vaccines’ reactogenicity, it’s hard to imagine participants and investigators could not make educated guesses about which group they were in. The primary endpoint in the trials is relatively subjective making unblinding an important concern. Yet neither FDA nor the companies seem to have formally probed the reliability of the blinding procedure, and its effects on the reported outcomes.
Nor do we know enough about the processes of the primary event adjudication committees that counted covid-19 cases. Were they blinded to antibody data and information on patients’ symptoms in the first week after vaccination? What criteria did they employ, and why, with a primary event consisting of a patient-reported outcome (covid-19 symptoms) and PCR test result, was such a committee even necessary? It’s also important to understand who was on these committees. While Moderna has named its four-member adjudication committee—all university-affiliated physicians—Pfizer’s protocol says three Pfizer employees did the work. Yes, Pfizer staff members.
Vaccine efficacy in people who already had covid?
Individuals with a known history of SARS-CoV-2 infection or previous diagnosis of Covid-19 were excluded from Moderna’s and Pfizer’s trials. But still 1125 (3.0%) and 675 (2.2%) of participants in Pfizer’s and Moderna’s trials, respectively, were deemed to be positive for SARS-CoV-2 at baseline.
Vaccine safety and efficacy in these recipients has not received much attention, but as increasingly large portions of many countries’ populations may be “post-Covid,” these data seem important—and all the more so as the US CDC recommends offering vaccine “regardless of history of prior symptomatic or asymptomatic SARS-CoV-2 infection.” This follows on from the agency’s conclusions, regarding Pfizer’s vaccine, that it had ≥92% efficacy and “no specific safety concerns” in people with previous SARS-CoV-2 infection.
By my count, Pfizer apparently reported 8 cases of confirmed, symptomatic Covid-19 in people positive for SARS-CoV-2 at baseline (1 in the vaccine group, 7 in the placebo group, using the differences between Tables 9 and 10) and Moderna, 1 case (placebo group; Table 12).
But with only around four to 31 reinfections documented globally, how, in trials of tens of thousands, with median follow-up of two months, could there be nine confirmed covid-19 cases among those with SARS-CoV-2 infection at baseline? Is this representative of meaningful vaccine efficacy, as CDC seems to have endorsed? Or could it be something else, like prevention of covid-19 symptoms, possibly by the vaccine or by the use of medicines which suppress symptoms, and nothing to do with reinfection?
We need the raw data
Addressing the many open questions about these trials requires access to the raw trial data. But no company seems to have shared data with any third party at this point.
Pfizer says it is making data available “upon request, and subject to review.” This stops far short of making data publicly available, but at least leaves the door open. How open is unclear, since the study protocol says Pfizer will only start making data available 24 months after study completion.
Moderna’s data sharing statement states data “may be available upon request once the trial is complete.” This translates to sometime in mid-to-late 2022, as follow-up is planned for 2 years.
Things may be no different for the Oxford/AstraZeneca vaccine which has pledged patient-level data “when the trial is complete.” And the ClinicalTrials.gov entry for the Russian Sputnik V vaccine says there are no plans to share individual participant data.
The European Medicines Agency and Health Canada, however, may share data for any authorized vaccines much earlier. EMA has already pledged to publish the data submitted by Pfizer on its website “in due course,” as has Health Canada.