Medical Image Analysis
A conversation with a mathematician about the problems of diagnostics using machine learning
Within the framework of the Knowledge Bank project, created together with Sberbank Corporate University and dedicated to modern technologies, mathematician Andrey Krylov talks about the specifics of processing and analyzing medical images – using neural networks and not only.
Analysis of biomedical images is an urgent topic related primarily to computer diagnostics.
We hope that mathematical methods and machine learning in the future will help to significantly simplify and speed up the diagnosis of diseases, especially in the early stages. Diagnoses will be more accurate, they will be established in a short time, which means that there will be more chances to preserve health and life. The breakthrough in image recognition that occurred around 2010 could mean that a lot will change in the analysis of medical images, but not everything is so simple.
Background and problems of analyzing medical images
As soon as the first computers appeared that could somehow work with images, that is, in the 80s of the XX century, the idea also appeared that they could be used to automatically analyze medical images. Representatives of various fields of science were engaged in working with medical images. But since the first computers worked very slowly and could open a small picture for a minute, or even longer, they could not yet be fully used in this area. Computers lacked performance, it was impossible to process large amounts of data on them, they worked very slowly.
Another direction in the processing of medical images is associated with the production of special medical equipment. The creators of these devices use quite powerful data processing programs in their devices. And the problem of analyzing medical images is connected with the fact that it is almost impossible to get raw data. The medical device has already processed them, and what we eventually see on the screen is the result of some kind of filtering, improvements, and so on. As a result of pinching the image through the same jpeg, a lot of information may be lost. Therefore, for a full-fledged computer analysis, raw data that has not been processed by other programs is needed.
Areas of application of computer analysis of medical images
Computer analysis of medical images is applicable in literally all areas – from ophthalmology to MRI. The study of fundus images is especially popular now, since this is the only place where you can see the vessels noninvasively. In addition, the first sign of diabetes – diabetic retinopathy – can be found just on the fundus. These studies are now being actively invested in the West, as the problem of diabetes is becoming more urgent.
A very wide range of tasks in the analysis of medical images is associated with dermatology. In general, all areas where there are images are being investigated. Each disease is a separate area of research. Therefore, from the abundance of diseases and various medical devices, there is a huge variety in these studies. For some analyses, especially for ultrasound and a number of other technologies (MRI, computed tomography), it is very important on which device this is done – algorithms are written for each specific model and its modes.
With the help of the analysis of biomedical images, you can investigate anything – from a hip fracture (which we did with our Singapore colleagues) to 3D modeling of teeth. Dentistry is especially interesting: with the help of modeling, the patient can see what kind of teeth he will have, how they will change from week to week if braces are put on – these processes can be simulated.
There are equal opportunities for research in all areas of medicine, and there is no one in which work would be carried out most actively. But the most priority direction is still the radiation diagnosis of the brain. In Russia, this is not very actively done, especially for neurodegenerative diseases, except for the diagnosis of Parkinson's disease, whereas in the West a lot of attention is paid to Alzheimer's disease.
Features of medical image analysis
The specifics of processing and analyzing medical images are primarily related to the need to work with physicians. But there are practically no medical researchers in Russia, there are only rare exceptions, for example, the Petrovsky RSCC. All our institutions are primarily focused on the treatment of patients, not on research. Therefore, doctors are very busy. They are ready to meet with data analysis specialists and give some data. But if it is necessary to conduct research on expensive and complex equipment, then this contradicts practice, the need to pay for such research and the use of these devices, and so on. In some areas, such as ophthalmology, this is sometimes easier.
Research is being conducted using ultrasound, color Dopplerography – the study of blood flows. There are a lot of tasks, and they are all interesting. Each individual study allows you to conduct research, to be published in decent journals. In some topics of data analysis, for example, in data compression methods, there is now a huge competition, it is already difficult to come up with something new. And there are many narrow areas in medicine that can be explored with the assistance of doctors and do good joint work.
Therefore, it is difficult to answer the question of which area of medicine is most computerized. But the most expensive are things related to MRI and CT. A separate area is surgery, that is, devices that perform operations.
Neural networks and medical images
In the last decade, there has been a breakthrough in image recognition, and since then neural networks have become firmly in vogue. But in the analysis of biomedical images, their role is ambiguous. This is due to several reasons.
Firstly, in my opinion, rapidly developing artificial intelligence technologies, of which artificial neural networks are a part, can still pose a threat to people. We do not fully understand what is happening in this black box, so it is difficult for us to control and predict the behavior of these programs. Technologies allow you to simulate voice, facial expressions, lip movement – this was demonstrated, for example, at the SIGGRAPH 2017 conference, where one of the speakers took a video speech by Barack Obama, replaced the text with a trained neural network and adjusted facial expressions to the new text. These technologies offer great opportunities, but they cannot always be used for good. You were given a neural network for research or work, but you don't even know what it was trained on, whether there are so–called bookmarks - undocumented opportunities. Or maybe she is trained so that in certain cases, if she works with bank data, the network will transfer money to some other account or transfer important data to the side.
There is a sharp breakdown in science now – a paradigm shift. And if earlier such changes took place so that scientists could somehow check what was happening, now we see that the same convolutional neural networks give good practical results, but they still remain a black box for us. And, returning to medicine, we have a very interesting situation. Medical data in many cases cannot be used and published, even if the patient's last name is not specified. In some countries, such as Taiwan, there are special ethics commissions that issue special permits for the use of even anonymous other people's medical data. In some areas – ophthalmology, retinopathy – there are standard databases on which you can check and understand something. In most other cases, researchers take some of their data from somewhere, do something with it using a neural network and then announce that they have an accuracy of results, for example, 90%, and this is higher than that of professional doctors. And the question naturally arises: how to check it? Answer: nothing. Previously, when writing an article, the author indicated which methods he used, and it was possible to understand how he managed to improve the results. And when describing the results obtained with the help of neural networks, data is provided extremely rarely, and therefore their origin and the reasons for obtaining a particular result raise questions. Accordingly, it is also impossible to independently verify this study, as well as to understand why a good result was obtained. On what data was the network trained? Which ones were she tested on? Reliability is often impossible to verify.
Neural networks in the scientific community
Of course, we use neural networks in the analysis of biomedical images. But the task of controlling the analysis of data obtained using neural networks remains complex and important. In medicine, this is especially important, since the lack of transparency in data analysis can eventually lead to a fatal error. In the scientific community, such caution can lead to strange results. Let's say you wrote an article about the processing of medical images using classical methods, you send it to a journal or to a conference, and reviewers ask: why were the results not compared with neural networks? Since 2012, in almost any field of science, you can find articles by authors (most often Chinese) who write about how they managed to achieve 99% of the result using neural networks. And no matter what you do, you cannot achieve such a result yourself, and it is very difficult to compete with Chinese scientists. Although it remains unclear on what data they trained and how they achieved this. 99% accuracy is a high result, significantly more than what even professional doctors manage to get.
The advantage of neural networks is that it is certainly a very good device for technological solutions. This is really a breakthrough in science, with the help of which it is possible to solve very advanced tasks, including in diagnostics. When you train the network yourself, everything is very clear. The same is true in medical images: you know what data the network is trained on and what to expect from it. And everything we get from other authors is very difficult to verify. It is extremely rare that something is done on a shared database.
To date, neural networks are used together with other methods and in the future, in principle, they are unlikely to be replaced. This is a useful thing, but it will never replace research related to mathematical methods. Actually, a relatively small number of specialists in this field are engaged in the development of neural network technology itself, and the rest are only users who take other people's networks and retrain them. And since there is little data in medicine, usually some kind of general neural network is taken and retrained on specific data. We use neural networks in the field of ophthalmology and diagnosis of Alzheimer's disease.
With neural networks, among other things, there is another danger: let's say we have pictures, the network works well on them, but if you add a little noise to them, especially if it is specially generated, invisible to the human eye, but sensitive to the neural network, then the network will give incorrect results.
Other methods of analysis
There are a lot of purely mathematical methods that are used in image analysis, including biomedical ones. They have been developing since the 80s of the last century. In them, image processing and analysis are based on the theory of signal processing, which has been developing since the middle of the XX century.
As for numerical methods, there are specifics related specifically to medical equipment. In medicine, there are a huge number of devices, each of which has its own physics, its own tuning features, different parameters and frequency domains. Some things we can't measure. For example, in MRI, we get a picture from the data using the inverse Fourier transform, and it can have a lot of different defects related specifically to physics – with the loss of frequency information. Therefore, the methods used are very different. And it is very difficult to single out a field of mathematics that would not be used here. Literally everything is used – from quaternions and topology to graph theory and statistical methods. And of course, there are a lot of methods related specifically to the physics of apparatuses. Of course, artificial intelligence methods, machine learning, convolutional neural networks, and so on are used.
Efficiency of biomedical image analysis
In different fields of medicine, the effectiveness of computer analysis is evaluated differently. There are common databases, for example, on eyes and retinopathy, where the situation is very good, the percentage of accuracy there is very high – above 95%. In other areas, it is more difficult to understand the effectiveness. A lot depends on what settings the doctor has set, because, for example, in ultrasound, the noise is not additive. This means that the results obtained in different modes are difficult to reduce to one indicator, somehow normalize and be able to compare them.
In general, the accuracy of computer diagnostics is slightly worse than that obtained by the most professional doctors, but better than that of average doctors. But the problem is that often one disease entails several others, and this greatly complicates the diagnosis.
In the processing and analysis of medical images, the ideal situation is as follows: there is a specific disease, there is equipment for diagnosing a certain medical modality (for example, ultrasound). We study sets of video data from the available database of patients and find with the help of machine learning some significant parameters corresponding to this particular disease. In addition, we have a database of not only images of one medical modality, but also complex medical histories, blood test data, and so on – in a word, the full picture. For a new patient, based on his medical video data, we calculate the parameters that we have determined to be significant for the disease in question, and the program does not give the doctor a diagnosis, but several, for example, five images of patients most similar in significant features. In this case, the doctor looks at the full medical histories of these five patients and sees various possible diagnosis options. At the same time, it will be much easier for doctors, even not the most highly professional, to work with this information, and the probability of errors will be reduced. In the case of issuing a diagnosis by the program (which is equivalent to issuing a medical history of only one patient with similar video data of the medical modality used), doctors of insufficiently high qualifications will simply agree with it, sometimes unaware of possible alternative diagnoses.
In the future, the creation of diagnostic programs should be designed specifically for complex diagnostics. Now the effectiveness of analysis and diagnosis depends not on the image, but on the disease that we are trying to identify. Let's say diabetic retinopathy is diagnosed well, but with glaucoma everything is not so obvious.
The prospects
I do not think that in the next ten years it will be possible to make any global progress in complex computer diagnostics - at least, so much so that these programs can replace even mid–level medical personnel. As already mentioned, there is not only one disease: one disease always entails other complications. It is extremely rare that only one organ is ill. Therefore, so far in the foreseeable future, it is unlikely that we will have any conditional boxes that can be connected to a person, and they will immediately give him a full diagnosis and prescription. But progress, of course, will be every year: diagnostics in remote regions will improve – precisely due to computer technology. In general, this is a big challenge – multimodal processing of medical images: imagine that you need to take an MRI, CT, X-ray and ultrasound from the same patient and give a comprehensive diagnosis using these four images. But slowly progress is being made, and the results are getting better.
About the author:
Andrey Krylov – Doctor of Physical and Mathematical Sciences, Head of the Laboratory of Mathematical Methods of Image Processing, Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University.
Portal "Eternal youth" http://vechnayamolodost.ru
02.10.2017