From politics to evidence-based medicine

Navalny claimed that the drug had not passed proper clinical trials, and it was not written about in respected scientific journals. In response, Chubais provided links to publications with the results of drug testing. Medusa asked the doctor of the Tarusa Hospital, Artemy Okhotin, to tell how the effectiveness of new treatment methods is evaluated, and how convincing the studies presented by Chubais are. 

Such an argument does not work even for a doctor who observes dozens of patients – "kagocel helps my patients." Many diseases go away by themselves or flow in waves: the state of health improves, then worsens. In these cases, any medicine will "help", even a placebo (dummy), especially if the patient believes in the drug. This is called the placebo effect. 

The standard of evaluation is a double-blind randomized placebo-controlled trial. This is a study in which the drug is compared with a placebo, and neither the doctor nor the patient knows who exactly is receiving the drug under study, and who is an indistinguishable outwardly dummy.

Previously, it was considered quite acceptable to conduct a study in which all patients received a new drug. The erroneous logic of such studies is well conveyed in a quote attributed to Galen (II century AD): "All patients who took this remedy soon recovered, except for those who did not benefit from it – they died. Hence it is obvious that this remedy helps in all cases except hopeless ones." The understanding of how dangerous the misconceptions that arise from the simple observation of the effect of drugs came in the 1960s and 1970s. When placebo control was used to evaluate drugs, it turned out that many conventional remedies not only do not help, but also harm. For example, the CAST study showed that drugs used in myocardial infarction to stabilize the heart rate do stabilize the rhythm, but increase mortality.

If the doctor doubts the new drug, he may subconsciously refer lighter patients to the drug group. The attitude of the doctor and the patient to the drug will also affect the assessment of its effectiveness, especially when evaluating such informal indicators as general well-being or the severity of cough. Therefore, neither the doctor nor the patient should know who is taking the medicine and who is taking the placebo. This allows the doctor to assess the condition without knowing whether the patient has received the drug. 

The results are discussed, rechecked and criticized. Analysis and criticism of clinical trials make up a significant part of publications in good medical journals. The principles of evidence–based medicine are developing, what we have named are the basics, without which they will not read the work and will not take it to a decent magazine. But not only the research methodology is important, but also its statistical significance, which excludes the randomness of the results.

Let's say we have invented a way to throw only tails in the heads-or-tails game. If we make three throws and get tails three times, the probability that we were just lucky is 1/8. Such a result will not be accepted in a medical journal, the probability of a case is too high. If we flip a coin six times and get tails each time, the probability of randomness is only 1/64 – this is already suitable for publication. But treatment methods rarely give the desired effect in 100% of cases, so large samples are needed to obtain statistically significant results. For example, in order to show that oseltamivir (Tamiflu) accelerates the recovery of influenza for a day, it took more than 700 patients to be recruited into the study.

This problem is called publication selectivity. If we repeat a series of experiments with a coin many times, sooner or later we will get six tails in a row and we can publish such a result. No one will know about unsuccessful experiments, which will create a false impression that the method is effective. To combat the selectivity of publications, a preliminary registration of clinical trials was invented, requiring the publication of any results. Interestingly, studies with negative results are often published in English-language journals, whereas it is not accepted in Russian-language publications, which greatly undermines their credibility.

Among the most high–profile such publications in recent years is the COURAGE study, published in the prestigious New England Journal of Medicine. It showed that coronary angioplasty, a procedure for restoring blood flow in the arteries of the heart, does not reduce the risk of heart attack and death in stable patients, they can be treated with pills. The study has become the main cardiological news for many years.

Statisticians should take part in the preparation of publications. Before publication, the article is reviewed in the editorial office of the journal (such journals are called peer-reviewed). However, publication of erroneous or inaccurate results is possible. Therefore, medical publications are subjected to rather harsh criticism by the professional community. If the criticism is sufficiently convincing, researchers may be required to provide primary data. Another level of verification is the state supervisory authorities (the Ministry of Health in Russia, the FDA in America), but they usually rely on the same professional community in their conclusions.

At best, you can keep track of your narrow specialty. But professional organizations come to the rescue, which examine existing data, evaluate their reliability and publish practical recommendations. They indicate the degree of validity of a particular recommendation.

The recommendations describe standard situations in which it is really possible to act according to the instructions. But you need experience and knowledge to understand what instructions to follow at the moment. In addition, not all patients fit into the standard. Often, in order to understand what will help the patient, the doctor needs to analyze the published data himself. The ability to work with data is as important a skill of a doctor as talking to a patient, recognizing symptoms or performing therapeutic manipulations. 

But drug manufacturers sometimes try to influence the results of research. Therefore, research sponsored by the manufacturer is believed less. When such an influence is revealed, a scandal occurs, from which not only the firm suffers, but also the authors of the research. Such a case was with the analgesic drug Rofecoxib from Merck. Studies have shown that the drug slightly increases the risk of heart attack, but the authors presented these data in a very innocent light. When it turned out that this was done intentionally, and some of the data was also hidden by researchers, the company had to pay several billion dollars and stop selling the popular drug. That is, not everything has been bought: where there is a judicial system and a professional medical community, you can fight the pharmacological mafia – just like any crime.

So far, no publications have appeared in respected medical publications confirming or refuting the effectiveness of kagocel. In the Medline system, a database of publications in all significant biomedical journals, there is not a single reference to clinical trials of the drug. The studies cited by Anatoly Chubais have been published in rather marginal publications. Most of the articles were written by the same authors, some of the research is not available, and there were small samples in the available publications and, most importantly, they did not indicate the statistical methods used. It also does not contribute to trust that the results of these studies have not been critically analyzed by third-party authors. However, the situation with many over-the-counter antiviral drugs is no better. For a doctor who is used to focusing on the principles of evidence-based medicine in his practice, kagocel simply does not exist yet.

---

From the editorial office:

We recommend reading a much more detailed and complex article on the same topic – "Evidence–based medicine as a means of promoting medicines" and a bone-by-bone analysis "Im- Kagocel Name. The answer to Anatoly Chubais", published on Alexey Navalny's website in continuation of the discussion.

Portal "Eternal youth" http://vechnayamolodost.ru

02.07.2015