AI for doctors

Medical applications of artificial intelligence

For illustrations, see in the video recording of the lecture in the original article.

What do we mean by artificial intelligence? What is common do the axons of the giant squid and AlphaZero? How does Watson fight cancer and why is he criticized? This is told by Sergey Markov, a specialist in the field of machine learning and the founder of the XX2 century portal. From the lecture, you will learn how artificial intelligence is changing medicine today, what has been achieved over the past ten years, and what tasks have yet to be solved.

Sergey Markov. First, before talking about the medical applications of artificial intelligence, and, in principle, about the applications of artificial intelligence and machine learning technologies, I want to spend a few minutes to determine what artificial intelligence is, because the problems with this definition are that initially, having arisen among the computational of course, it quickly got into pop culture and, accordingly, has already undergone such transformations there that now, of course, if you stop a person on the street and ask what artificial intelligence is, you will probably hear very different strange things from him. Therefore, I will first of all say what we mean by artificial intelligence (we mean specialists who are engaged in creating machine learning systems). Alan Turing was a man, in fact, distrustful of philosophers, and when he defined the term artificial intelligence, he tried in every possible way to avoid defining the concept of intelligence. Because if we define the concept of intelligence, then we can get bogged down in discussions about it for decades, so, in general, experts decided to start here from the concept of an intellectual task. That is, there are intellectual tasks – these are tasks that people traditionally solve with the help of their human intelligence. And if we create a system that is designed to automate the solution of such a task, then such a system is an artificial intelligence system. Well, everything seems to be fine, and there is almost nothing to argue about in this definition, but here you can see that this definition is very broad, because if you think about it, people perform a lot of different tasks with the help of intelligence itself, for example, oral counting tasks. If we add or subtract numbers, then, in principle , it turns out that we use our brain at the same time, we use our intelligence at the same time. And in this sense, a primitive system that automates the task of counting, it will also be an artificial intelligence system in accordance with this definition. And so, in order to draw certain boundaries here, after all, by spreading flies and cutlets into different plates, another very important division was introduced, the differentiation is the differentiation of artificial intelligence systems into weak artificial intelligence, or as it is called applied artificial intelligence, and hypothetical - not yet existing systems of universal artificial intelligence, or sometimes they are called systems of strong artificial intelligence – Artificial General Intelligence – there is such a concept. What is the difference between these two types of artificial intelligence? Weak artificial intelligence (or applied artificial intelligence) – this is a system designed to solve one specific task. That is, you have a program that plays chess well, it solves this problem well, but it can't drive a car, it can't recognize images, it can't recognize speech, and it can't perform another thousand, hundreds of thousands of intellectual tasks – this is an artificial intelligence application system. And in this sense, all the artificial intelligence systems that we have created so far are applied artificial intelligence systems. And if we talk about universal artificial intelligence, it is a kind of hypothetical system that will be able to solve an unlimited range of intellectual tasks, just as the human brain does. That is, any task that is within the power of human intelligence, it will be within the power of such a universal system. Well, among the weak and applied artificial intelligence systems, we again say that there is such an informal border, that we will not consider any simple trivial intellectual tasks, after all, systems that are designed to automate them, artificial intelligence systems. Well, it's more a matter of taste, because, say, two centuries ago, the task of automatic counting using a machine was extremely difficult from a technological point of view , and in principle, of course, the first mechanical arithmometers there were the pinnacle of technology at that time, and then no one would turn up their noses from such a task as automation of counting. Well, today, of course, it doesn't seem to work out very well to talk about a calculator as an artificial intelligence system. In principle, what is important to understand? It is important to understand that the progress that we have seen over the past 5-10 years in this area is progress, first of all, in the creation of applied artificial intelligence systems. And this progress is mainly due to the fact that a number of intellectual tasks that have been unavailable for artificial intelligence systems for a long time, that is, we could not, in general, create a system that would effectively solve such a task, these tasks have been solved. For example, this is a victory over a person in go – a rather hyped task, because very often skeptics liked to cite go as an example of such a task that machines solve poorly, significantly worse than humans, and where progress is extremely slow. And, in general, even 10 years ago , there was serious talk that go either could not be solved with the help of existing approaches, or it would be very long before it would be possible to create a system that would play go. In fact, there is a huge range of such tasks, about which the first spring of artificial intelligence broke its tooth at the time , which began in the late 50s – early The 60s ended, in general, with the collapse of many hopes that we observed in the late 70s. And for a long time the system artificial intelligence has been forgotten, in fact, the very concept of artificial intelligence has become such a bit of a garbage term in the mass consciousness. And, it would seem, for a long time such models as neural networks were forgotten, only narrow specialists knew them, somewhere, well, when I received my IT education, we had an elective course where we were told that there are such neural networks, but it was not considered as mainstream some models capable of solving serious problems. But we are now witnessing the dawn, which first of all began as a technological dawn, continued as a media dawn, and at the moment, the dawn of investment, because quite serious money is being invested in the development of artificial intelligence machine learning systems. And they were largely rehabilitated in the eyes of both technical specialists, investors and the public.

What are these successes that we are seeing connected with in general? In my opinion, there are three very serious prerequisites for why this happened right now. The first prerequisite of this kind is an increase in the computing power that we have. It's no secret that Moore's law is still working, and computing power is growing at such an exponential pace, so our machine is getting faster and faster. What was extremely difficult to do two decades ago, the grid sizes that we could afford (they were very small) – now we can create larger models, we can use more computing power. At the same time, there has been progress not only arithmetic, in the sense that the central processors have become faster. But in fact, there is another reason – we have learned to use tensor processors, which now have video cards embedded in them – this is specialized hardware that allows a number of tasks to be solved significantly faster than they are solved on central processors. And looking ahead a little, I will say that now almost all the hardcore giants have got involved in the race to create specialized equipment for training neural networks. That is, these are no longer as such video cards that we use to train the network, but these are specialized boards that are designed primarily for training neurons. The second reason is the creation and accumulation of large amounts of information on which we are able to train machine learning systems. If in the late 70s you wanted to create an artificial intelligence system capable of understanding something in natural language, then you would have come across such a very serious problem - that you had very few electronic text corpora. That is, of course, you could take a bunch of graduate students or undergraduates, force them to type "War and Peace" for a whole year for tests and exams , and, probably, they would even do it, but a lot of work would be spent, at the output you would actually get a very modest set of texts on the scale of the whole variety of the language in general. That is, this text would not really even closely reflect all the linguistic features, all the richness of semantics, all the richness of vocabulary that exists in the Russian language.

Thanks to the advent of the Internet, the amount of information that humanity possesses is now, according to various estimates, doubling in 2-4 years . That is, it means that, in general, the volumes of information accumulated to date are, of course, enormous. And today, in terms of natural language, in terms of images, in terms of videos, in terms of sounds, huge, huge amounts of information have been accumulated that our predecessors did not even dream of 20-30 years ago. Well, not the last contribution, in general, industrial automation has made here, because we have been actively using a computer in production management since the late 80s, in managing a variety of processes in society, and all this also leads to the accumulation of huge amounts of data, on which, again, systems can learn. Well, for example, for example, this is the company I work for, it has been carrying out about 250 thousand conversations with people by phone every day since 2013, and they keep all the records of these conversations. That is, 250 thousand negotiations a day since 2013 is a huge array of information from the point of view of linguistics, from the point of view of phonetics, from the point of view of vocabulary and much, much more. On such an array, you can actually train the system to understand a lot. Well, the third, but not the least important, is the creation of new models. Generally speaking, most of the models we are dealing with today, say, convolutional neural networks or specific types of recurrent neural networks, for example, LSTM networks – they were invented somewhere on the border of the 90s and 2000s. But no matter how few specialists have worked in this direction so far, of course the technological gap was very large. That is, for example, in 1999, a key work on convolutional neural networks was published by Jan Lekun, where he showed that such models are able to recognize images very well. But the attention of the public was seriously attracted to this model only in 2011 with the work of Hinton [Jeffrey Hinton] around ImageNet, when neural networks on the ImageNet corpus showed a huge gap from the competitive image recognition models at that time.

This slide is not needed to read it, it is needed to suppress it, so each rectangle on this slide is a separate area of application of artificial intelligence machine learning technologies. In principle, three dots are drawn here in the lower right corner, because this is what we managed to come up with and remember quickly enough, and if you sit longer, you can probably increase the number by 2 or 4 times.

Well, a little bit about neural networks and, generally speaking, where this model came from. In general, there is such an approach as bionics in science. Bionics is like this, in general, if everything is bad for you, if you don't have any good engineering solution, but there is a system like the one you are designing in nature, then you can try to steal this solution from nature, somehow peek. Leonardo da Vinci, for example, when he was thinking about creating the first heavier -than-air aircraft, he actively studied the movement of a bird's wing. We know that in his drafts there are a lot of sketches, a lot of sketches, where he is trying, in fact, to comprehend this mystery of flight, in order to then reproduce it already in a mechanical design. Well, I must say that bionics is usually the very first steps, the very first movements of technology in a new direction, and most often technological systems that eventually become the main solutions in technology for a long time, they are quite far from their prototypes from wildlife. That is, we, in general, see that airplanes do not flap their wings unlike birds and, in general, use different principles. But, nevertheless, at the earliest stage, at the very first steps of observing wildlife, it can push to create at least the very first working models. And in this sense, when creating artificial intelligence, the very first approach that appeared, it was also the same bionic approach, in the late 40s McCulloch and Pitts worked on a model of an artificial neuron, and this work was, in general, quite in close connection with neurophysiologists. The fact is that neurophysiologists, of course, turned out to be pioneers here, and they thought about the work of nervous tissue, about the work of the brain much earlier than specialists in computational mathematics. Well, there is nothing surprising in this, but the first model of an artificial neuron was created in the late 40s and, in fact, the first prototypes of neural networks are already appearing in the 50s.

What did you manage to spy in wildlife? In living nature, it was possible to see that the brain is such a tissue consisting of nerve cells called neurons. And, in general, you can notice that each neuron has one long outgoing process, which is called an axon, and a certain number of such short input processes, which are called dendrites. And, accordingly, the axons of neurons connect to the dendrites of other neurons. And so the place of contact between the axon and the dendrite is called the synapse. In general, there is such a synaptic gap into which various neurotransmitters can be delivered – these are substances that change the characteristics of the signal as it passes through the contact. Accordingly, the observation of this tissue suggested that, in general, such a simple model can be created, which is nothing more than a weighted signal adder. That is, if we look at a single neuron – it has these dendrites, we will compare some number, some weight coefficient to each synapse, and here is the signal that comes through the axon to this dendrite, we will multiply by this coefficient. And then we will sum up all these works and transform them with the help of a certain transitional function. But I must say that in the first neural networks in general, this was not a smooth function, but it was a Heaviside function, so -called. She looked very simple: but if the received amount is greater than zero, then a single, single signal is transmitted to the output. And if the sum is less than or equal to zero, then, accordingly, no signal is sent to the output , well, a zero signal is sent, as it were. Accordingly, the weights can be both positive and negative there, that is, here we got some number for input, multiplied it by the first coefficient, here we got a number, multiplied by another coefficient, and so on. And, accordingly, they added it up. If the sum turned out to be greater than zero, then we transmitted a single one along the axon to the next neuron, if not– then we did not transmit it. However, it is not very convenient to teach such networks, and therefore , over time, they came up with the idea that here you can behave a little more cunningly: you can make such a function in which the transition will be smooth, smoothed, it can be a logistic function in this case, sometimes a hyperbolic tangent was used. In general, such a very simple mathematical model, which, as it turned out, works quite well, and it turned out that such networks created from many, many neurons, they can actually cope with a number of tasks, for example, pattern recognition, classification tasks, and so on and so forth. These are just the people who did this work, Warren McCulloch, Walter Pitts, Donald Hebb. And, in fact, their artificial neuron. The first incarnations of this model itself, oddly enough, were not created on the basis of digital technology that existed at that time, because digital machines were then too slow to emulate any large neural network in real time. Therefore, there was such a brave man Frank Rosenblatt, and he, in general, literally built the first artificial network of sticks and tape practically. This is such an array of artificial neurons, these are the wiring that he has piled up here, these are artificial synapses. You just took such a wiring and connected some neuron, for example, here and here, stuck the wiring into the corresponding sockets, and you got such a synaptic connection. And all this was very loudly called the Mark-1 neurocomputer. In general, this enterprising guy even managed to sell a number of such devices to banks. Well, some tasks were solved even with the help of such hardware. This is such a distant, distant great-great-grandson of this kind of device, this is a neuromorphic processor TrueNorth of IBM. If you look at this piece of hardware, there are 1 million artificial neurons and 256 million emulated synapses in it. But this, by the way, is the previous generation, in my opinion, now there are 4 million already, too, the same piece of hardware in the same form factor. As for the neurophysiological roots of this model again, it means that the fact is that everything I've told you before actually has almost nothing to do with how the nervous tissue actually works. Because the model that McCulloch and Pitts created, of course, is very far from how the signal actually propagates in a real brain. But how the signal propagates in the real brain, these are the questions that such wonderful people as Andrew Huxley and Alan Hodgkin dealt with in the 50s. And their main, in general, model object was the brain of a squid. Why squid? Because squids and cephalopods in general have very large-scale neurons. That is, if you look: there is a single neuron of the squid brain on the slide, this is a long thread this is, strictly speaking, the axon of this neuron. And due to the fact that it is such a large–scale object, it is quite easy to study, that is, you just take a voltmeter and, please, call the whole network - which, in fact, was what Hodgkin and Huxley were doing. And as a legacy from Hodgkin and Huxley, we got the so–called Hodgkin–Huxley model, a model whose heirs are now the basis of a separate large direction called impulse neural networks - spiking neural network. These are neural networks, which, in general, have a fair degree of biological reliability. You can see what happened, generally speaking, in terms of reliable models of the work of nervous tissue. It all began at the very beginning of the XX century with the works of Louis Lapik, who used nerves in the legs of a frog. And he actually created the first model, which was called "integrate and work." And, as a matter of fact, here we see that just at the beginning of the zero years, models are actively being created, which in turn are refinements of the Louis Lapik model. And as for the axonal part, this is the Hodgkin–Huxley model. And here again , the thing is that Hodgkin and Huxley worked with the nervous system of squids. In squids, generally speaking, the nervous system is an example of convergent evolution, that is, it is actually a very indirect relative of the mammalian nervous system, and therefore not all the conclusions that were made by Hodgkin and Huxley are correct in relation to the brains of rats or humans, for example.

What are scientists doing today to better understand how the brain works? One of the most probably beautiful, beautiful and relatively small projects is the EyeWire project, created by scientists from MIT (Massachusetts Institute of Technology). In general, his story begins with the fact that there was a mouse Harold, and then a mouse Harold is dead. They say that even by his own death. After that, his brain was removed and sliced into micron layers, passed through an electronic scanning a microscope and, accordingly, a huge number of slices were obtained , actually as thick as a single cell. And now the task of this EyeWire project itself was to recreate the fully three-dimensional synaptic structure of this mouse's brain. But, in general, even rough estimates showed that those three dozen people who worked in this project were not even close enough to decipher all these pictures in any reasonable time. Therefore , a diabolical plan was devised, which consisted in the creation of an online game, EyeWire, in which you can register and compete with other players in which of you is better able to color slices of the mouse brain. You are given such cubes, you color them with markers. If you have painted correctly, then you get a lot of points. If you have painted incorrectly, then very few points are awarded to you. Well, then there is a table of records, and you can, in general, be proud of the fact that you scored a lot of points. But, in general, due to the fact that the game is certainly somewhat, as they say, creepy, not a lot of people play it, but at least enough people have played it to create a training array of colored slices in order to then train an artificial neural network convolutional to perform this coloring automatically.. That is, here is such a beautiful two-stage project, as a result of which a tool was created for automatic decoding of such sets of slices and, accordingly, visualization of a piece of the retina of the mouse Harold.

The biggest project, of course, which is dedicated to deciphering the processes occurring in the mammalian brain is the Blue project Brain, which started back in the early noughties, and the ambitious goal of this project is to create a working electronic copy of the human brain, a working biologically reliable model. And the timeline of the project initially assumed that by 2023 the first model of this kind would actually be obtained. However, we were not promised that it would work in real time. Apparently, it's still there tens of thousands of times the delay compared to the real brain. What, in fact, was done during this project? Firstly, a rat neocortex column of 10 thousand cells was recreated, and it was shown on this column that the signal in it really propagates in the same way as in a real neocortex column. That is, the simulation data coincided with the propagation of the signal in the present column. In July 2011, they made a meso-closure in 100 columns of the neocortex, that is, a million cells. And, in general, by 2014 we were promised a working copy of the rat brain. But so far the results have not been published. It's hard to say whether the specialists are behind schedule, or maybe they have already succeeded, they just don't want to show it to us. But if you look at the publications that the project participants produce, you can see that in 2015 they made fundamental discoveries that, apparently, require some reworking of what has already been done. A secondary network of signal propagation in the brain through astrocytes of glial tissue was discovered, that is, it turns out that the signal, generally speaking, spreads not only through neurons, through synapses, but astrocytes can also participate in signal transmission. Accordingly, there is such a beautiful publication on this topic. We'll see, we'll see, at least officially, the postponement of this deadline until 2023 has not yet been announced. These works on the analysis of the structure of the retina, strictly speaking, are not EyeWire, of course, but earlier works that were done in the same field, they inspired computational mathematics specialists to create new computational neural networks, their new varieties are convolutional neural networks. A convolutional neural network is a neural network that was created just under the impression of analyzing the structure of the retina. One of the curses of neural networks in the 70s was that well, well, we came up with a model by itself, we came up with how a single neuron works, we came up with how to connect neurons to each other. What topology should a network have that effectively solves a particular problem? Here are exactly which neurons, with which other neurons should be connected? We have learned to teach networks more or less normally, especially with the advent of the error back propagation algorithm. Now, if we already have a ready-made synaptic structure, we more or less know how to teach it using gradient methods. But which neurons to connect with which? And there were actually a lot of different searches on this topic, jokes, jokes, attempts to randomly generate these connections there. It is quite obvious that not every neuron in the human brain is connected to every one. That is, we have 86 billion neurons, approximately 150 trillion synapses in the adult brain cortex alone, but this is much less than 86 billion squared. That is, not every cell is connected to every cell at all. What a good idea came to the idea in the late 90s that you can actually use the principle of such a Lego constructor. That is, you can come up with layers that are built according to certain regular patterns. That is, for example, a layer in which each neuron of the previous layer is connected to strictly four neurons of the next layer. And the neuron next to it is also connected to four neurons of the next layer, and the neighboring one too, and so on and so on. And it turns out such a regular structure, that is, one layer of the neural network is described as just a repeating regular pattern. And due to the fact that we parameterize these layers, we get a set of such building blocks, from which we can assemble a neural network and, selecting the parameters of individual bricks, and selecting sets of these bricks, we accordingly create an effective model in order to solve certain tasks. But, and actually specifically the retina, she suggested that there really is such an operation as convolution, when some small region, some small area of photosensitive receptors, from it the next axon explores there into 2, 3, 4, 5 layers, and these connections also obey a certain regular law. Having recreated such networks, we saw that they are very effectively able to recognize images. To date, the heirs of this Lekunov network, they have overtaken man in the accuracy of pattern recognition. This is the problem I started with Lecun handwriting recognition MNIST-array.

That's what happened with pattern recognition. You see 2010, ImageNet is the largest image recognition competition. You have several million images, and you need to determine the presence of certain objects in the picture: a bird, a person, and so on – a large set of objects. In 2010, the error on this ImageNet array was 28% for the best model that existed at that time. And so, as a matter of fact, the first convolutional neural networks come here in 2011, and we further see that in 2014 this indicator drops to 7%. In fact , in 2015 it drops to a little over 4 percent. It is generally believed that people make about 5% of mistakes on this set. That is, thus, in 2015, parity with people in pattern recognition was achieved for the first time. Here is such an incomprehensible diagram, it actually shows that the recognition accuracy is here on the Y axis, that is, the higher the circle is, the more accurately the model recognizes the picture. On the X-axis, this is the number of millions of operations to compute a single network. That is, we crammed into it a picture of how many millions of operations we have to perform in order to get some kind of solution out of it. And, accordingly, the number of synapses, relatively speaking, in the network is the diameter of this circle. We see that modern models can be limited to a relatively small number of synaptic connections, while demonstrating higher accuracy than older models. In fact, the pursuit here, of course, is for optimal architectures of this kind of artificial retinas. There are many tricks invented there, how such a network can be made more accurate. Well, that's what modern neural networks can do – they can solve such combo tasks, that is, they can sign to you with words of natural language that is drawn in this picture. It doesn't always work out exactly, but in fact, in most cases, it's very good. Well, even when he makes mistakes, then, in general, mistakes look quite human.

The situation with speech recognition is actually very similar . Here, recurrent neural networks performed well, with elements of convolution, too. In 2016, a research team from Microsoft showed on a certain chemically pure dataset the accuracy of speech recognition comparable to the accuracy demonstrated by humans. Of course, there is some criticism of this work, but in general we can say that the state-of-the-art technology today is such that even if parity is not reached, then at least it is very, very close. If you take my team again, one of the solutions that we were developing, there was a system for automating the call center operator. And for any practical purposes, in order to understand what a person said to you on the phone, the current speech recognition systems are enough, even despite the high noise level of networks, because there the signal passes through a bunch of distortions and so on. These are also some of the achievements of convolutional neural networks – generative models – over the past few years , neural networks have been used quite actively not only to recognize an image, but also to create new images. What is the basis of the idea? Here you have a network that recognizes dogs. We trained her on a large number of images of dogs. They taught me to distinguish pictures with a dog on them from pictures with no dog on them. And now, having taken some arbitrary image, having received for it, as it were, a certain assessment of the degree of dogness of this image, you are now trying to go from the end – to increase the degree of dogness of the image that entered the network. To do this, you simply go through the network from right to left, multiplying the signal value in each neuron in proportion to the weight coefficients of the convolutional cores. And on the first word you will get a picture with an enhanced degree of dogness. That is, everything that was remotely similar to a dog, now looks very much like a dog. This project was called Deep Dream, it resulted in a whole big direction, which is called Artistic Style Transfer, that is, the transfer of artistic style. You can teach the network the peculiarities of some artist's writing, and then transfer this style to any arbitrary image. Here are a few more examples of this kind for entertainment. Well, here is the picture that was submitted to the entrance, the picture on which the style was taught, and actually here. From these two images , this third one is obtained. That is, one image is enough to learn the style. If there are a lot of them, then so much the better. It is quite obvious that the same tool can be used for coloring black-and-white images, all the more so here everything is greatly simplified, because to train such a network, in fact, you do not need to mark up a dataset in any way. You just take lots and lots of pictures, discolor them, so you have, as it were, pairs, and then you teach this network to color pictures. In fact, if you think about it like that, what is the basis of what this network works on, well, it actually learns what color different objects are. That is, in fact, this is about the fact that after passing through millions of images, the network understands that human skin has such a shade that the crest of the rooster is red and so on and so on. That is, having seen a comb somewhere, in fact, having learned to recognize it, she thereby understands what color this object is. Where, again, these models are wrong, well, here is just an example, that is, the picture is black and white, this coloring is made by the network, and this is the original picture. That is, we see that the bag is actually so red, but here it turned out to be brown. Well, as it were, why shouldn 't the bag be of this color – there is no contradiction, as it were, to the objects that actually exist in this coloring book. This is my favorite project, of course. It turns out that using this approach, you can generate porn, because if you have a model for recognizing porn in pictures, and, in general, all major social networks, major content collectors, have this kind of model in order to distinguish what can be shown to children, what is not. You can then take this picture and this network and again also turn it inside out, forcing it to generate pictures with a given degree of porn. That is, this is a small degree of porn, these are such pastoral pictures, according to the network, and this is, in general, hardcore. It is clear that while the quality is not very specific in this project, but I think that the technology will improve, and soon the quality of porn will grow.

Now that I have galloped through Europe on what is happening in the field of artificial intelligence in general, I will tell you about the medical applications of artificial intelligence. The main, probably, at the moment, 8 areas in which artificial intelligence systems are used in medicine are the recognition of medical images. It is quite obviously an obvious task. We have a lot of X-ray images, CT, MRI and so on and so on, in addition to this we have data from electrocardiograms, electroencephalograms and so on and so on. All this, in general, can be described in one broad term - medical images. And, accordingly, the task of searching for certain objects of particular interest in these images is, of course, quite obvious, given that great progress has been made in the field of image recognition.

Analysis of medical texts. Well, medical text is again a very broad concept. Firstly, there is a huge corpus of medical research publications, that is, if we take some PubMed, there are millions of documents on it, which, of course, a person can hardly read manually in any reasonable time. Accordingly, a system that is able to identify some important information in these documents, some important features and then generalize it on large amounts of information, it is, of course, of quite serious value. In fact, in addition to research texts, there is still quite a large array of such working medical information: these are medical histories, these are various other textual patient information, which again exists, of course, completely not in a standardized form. That is, usually, as if partial standardization exists within individual medical institutions or individual national health systems. And in this sense, let's say, the task of bringing together all this variety of patient information to some single comparable structure is, of course, a task for such intelligent data processing using artificial intelligence systems. It is unlikely that this entire array of information can be processed manually.

Medical interfaces. I have already talked about the successes in the field of speech recognition, in fact, great successes are observed in the field of speech synthesis and, accordingly, video communication. That is, there are models who can read text by their lips, who can determine a person's emotions from a video recording, who can even make some diagnoses from photographs – I'll tell you a little about this later . Accordingly, at the moment when you need to interact with a patient, interact with a person – somewhere this task can also be automated, that is, the task of, for example, the initial survey of a patient - it may well be performed by an automated system that will collect some primary information, perform some primary processing. The advantages, of course, of such systems are fabulous cheapness compared to how if we spend the time of a qualified specialist on such a task. That is, even in a call center, for example, here is an automated call center operator who simply follows a certain script in which there are 50-60 states, despite the fact that traditionally they save on call centers, these call centers work there in the Russian outback, and the rate of the call center employee there is 15 thousands of rubles a month, but the robot is still 4 times cheaper, that is, despite the fact that it is still such a technology bordering on experimental. Therefore, in some cases, people simply do not have access to normal medical care due to the fact that there are not so many doctors. Doctors are really a scarce resource because of the way budgets are distributed in our country, and not only in our country, but all over the world. Somewhere thanks to the use of such interfaces, it is possible to increase the availability of medical care.

Signal processing in bionic prostheses. This is a separate big direction. We have actually learned to create a lot of complex devices in order to restore the lost functionality of the body. Of these, the most complex, probably, can be called artificial vision devices. Generally speaking, the first artificial vision devices appeared paradoxically in the late 70s and were associated with the work of such a specialist as William Dobell. But this, of course, is an absolutely amazing story about how open-brain surgeries were performed in the late 70s , bundles of thin, thin electrodes were implanted into the visual cortex, 39, in my opinion, Dobell had electrodes in his original scheme. It was still a time when there were no digital cameras, when there were no fast computers. In fact, the analog camera was connected to the whole mainframe, which then "processed" this signal. Then the whole thing was connected to a person. Naturally, no one would approve of such operations in the civilized world, so Dobell took his patients to the uncivilized world, then Portugal was an uncivilized world. He took his patients to Portugal, they were operated on there, microelectrodes were actually implanted in the brain, and then he connected them to the car accordingly. And so people saw such flashes of light in their field of vision, they called them phosphenes. And for about 30 years, this technology was gradually improved, until in the early noughties, the first such operations began to be done on a commercial basis. There is a book by one of Dobell's patients, Jens Naumann, who tells his whole story. In the early 2000s, with the help of a vision prosthesis, it was even possible to read letters, but somewhere there 7-8 centimeters high, it was even possible to drive a car around the house, but so very slowly and carefully and under the supervision of other people, but nevertheless. This year , we had the first operation in our country to implant a vision prosthesis – this is such an Argus system, it is actually structurally simpler than the devices that Dobell used. But it's actually a long story. There is again, the prosthetics of hearing loss has already entered into widespread practice today – these are the so-called cochlear implants. Moreover, today this is already such an ordinary operation that children, for example, with hearing loss, are fitted with two cochlear implants at once in order to provide volumetric hearing. That is, it is considered that this operation is quite trivial. In fact , a wireless receiver is inserted under the skull, which transmits a signal, transmits it to the cochlea, and a transmitter is placed outside, screwed to a microphone, and a microprocessor that processes this signal and converts it into an analog signal, which the brain is then able to recognize. Accordingly, limb prostheses are also being actively created, prostheses with feedback, of course, which are capable of transmitting the nervous system back, informing about the texture of the object, its density, and so on. In order to process signals correctly, you actually need to process large amounts of data. In order to set up this kind of prosthesis, to adjust it to the nervous system of a particular person, to switch these channels of information transmission between the piece of iron and the wet part of the system – this also requires machine learning models, artificial intelligence systems. The more advanced they are, the correspondingly higher the quality of such prostheses we can provide.

Medical robots. Again, there are many examples here, I will also show several such projects now. This is also help for doctors somewhere , where, for example, medical personnel are very expensive, such systems are actively developing there. For example, in Japan there are specialized robots for carrying patients, because, again, it turns out that such a system, despite all its bulkiness and obvious technological complexity, is cheaper than highly paid medical personnel.

Data analysis of wearable devices. Probably almost every person today has a mobile phone, in fact there is a very valuable sensor in a mobile phone – an accelerometer, sometimes even a gyroscope, and oddly enough, this information that these sensors collect, it can be very useful from the point of view of diagnostics. It turns out that having such a device with you, some, let's say, dangerous conditions can be diagnosed. And this is actually very cool, and now such models are being actively developed that will help to carry out some preventive actions. Not to mention the fact that there are various kinds of specialized wearable devices, there are, for example, cardioflashes, that is, these are wearable devices that allow you to take a signal about the activity of your heart in real time, which of course is extremely useful considering that the main cause of mortality in our country is cardiovascular diseases.

Another direction is the search for new drugs. This is especially interesting due to the fact that great success has been achieved in principle in the creation of new substances using modern machine learning methods, in particular there is such a method of swarming particles. And here are his heirs, for example, such a wonderful specialist as Artem Oganov works in our country, who predicts the crystal structure of substances, predicts the properties of substances according to their formulas, and, of course, this direction has a very big prospect in terms of creating promising medicines.

And the most obvious direction that has been developing for a long time is automated diagnostics, expert topics for automatic diagnosis based on some large sets of information about the patient. This is the main thing. In fact, there are, of course, some additional areas there, but we will talk about them today.

What is important that we observe in the field of artificial intelligence economics, medical artificial intelligence? The years 2014-2016 were marked by several such very serious marages between companies specializing in machine learning and medical companies. We see that, first of all, large IT giants went into medicine, who felt the power of the tools that they created. Probably the most famous and the first story, which some even consider a false start, is Watson Health program and separately its big spin–off, which I will also say a little today, Watson for Oncology is a project dedicated to the diagnosis and therapy of cancer. "The Chan Initiative – Zuckerberg" biohab, dedicated to the search for drugs and the creation of a cellular atlas. Google founded the company Calico, which specializes in the fight against aging. A large biological section was opened by Intel. Microsoft announced that it would provide cloud services for drug search. Apple said it would focus on analyzing wearable device data in the health field. Bioepis [Samsung Bioepis] and Takeda [Takeda Pharmaceutical Company Limited] is a big project to create new medicines. Takeda is the largest Japanese pharmaceutical giant. And in general , 106 startups were opened in 2016, involving the use of artificial intelligence in various areas of healthcare.

Diagnosis and selection of the optimal treatment plan. Here is an example of the Watson system itself. In fact, Watson is some kind of application for a universal system for sentencing analysis (sentence analysis) of texts. It all started with the fact that Watson for Oncology a training array was prepared, a training sample was taken 100 thousand medical documents, that's about 15 million pages of text, and it all included 25 thousand medical histories, 300 medical journals and 200 medical textbooks. Accordingly, IBM has founded a number of joint projects with doctors to create the Watson system for Oncology. It's all like dry figures and what-when happened. What does it look like? That is, there are always people who hear about Watson for Oncology, they are interested, of course, to find out what it is for the end user. This is one of the Watson for Oncology interfaces. What do we see on it? This system recommends treatment plans to us, and each treatment plan, first of all, has a certain assessment there outcome – to what extent this approach is justified in this particular case. There is also an opportunity to give feedback right away, and doctors who use Watson have the opportunity to write comments there right away, if they either used this method of treatment there, or there have some opinion about this treatment plan - they are able to add this feedback here . There is also an opportunity to view clinical studies related to this case, that is, by clicking here on this tab, you will see a list of publications on the results of clinical studies related to this particular case. Firstly, medical images, that is, various scans, are submitted to the input of the system , if you have some X-ray data, plus some kind of such a large questionnaire that the patient fills out about his condition, some objective data. The results of various tests are also entered there, here are all the tests that you have passed, they are entered there, that is, it turns out such a large brick of information about the patient, on the basis of which Watson, in fact, issues this list of recommendations. As you can see, this is a system, in general, it is intended for doctors, that is, it is a decision support system. There are all sorts of enthusiasms and reviews about the work of this system, but you need to understand that, of course, its obvious drawback today is that it is sharpened for a very small number of diseases. That is, specifically 9 types of cancer, 6 now, 9 are planning to add more, so it seems like 80% will cover in frequency. But there is criticism, there is meaningful criticism, we will not analyze it in detail here, of course . For example, one of the areas of criticism is that the recommendations that Watson issues are very much influenced by specific consultants who advise Watson, and, in general, not everyone agrees that they are the benchmark in this area. Plus, of course, the project is very expensive and has been going on for a long time. However, there is an opportunity to take a peek to see how this system works. When they started their Indian project Manipal Hospitals, on the website manipalhospitals.com you can see, there is a demo version of just the Watson interface. This is about the turnover of a small such project, in which no giants work, but, in general, a very small startup.

Diagnosis by photo. Well, it sounds ridiculous, but in fact we are talking about those diseases that are really displayed on the appearance of a person, these are mainly chromosomal abnormalities, Down syndrome, for example, and a number of other chromosomal abnormalities. Here is a Face2Gene project. That is, you just upload a photo there and, accordingly, 7 thousand known genetic syndromes that affect the appearance of a person allow you to identify.

Well, here are the medical robots I was talking about, here, of course, you can't deny the humor to the creators of this project. The project is called ROBEAR. Such a pedobir who tolerates patients. The 140 kilogram design weighs, allows you to carry people very carefully and gently and touchingly on your hands. And, in fact, the project is already well-deserved, because the first prototype appeared in 2009, it was called RIBA. And, in general, today it is already in the replication stage. But this is a traditional problem, in fact , even the lowest medical staff in Japan are very expensive employees, so, of course, there is a search for such solutions. And what we have today in Japan will probably be in other places in 10-15 years .

This is another story, these are Da Vinci robots, about which many people have probably heard. Again, you need to understand that these are not exactly robots, that is, it is rather a system that is a surgeon's assistant, but it allows the surgeon to perform the operation alone where, in general, many other employees assist him under normal conditions . And this system is able to perform some surgical manipulations much more accurately than a person. That is, for example, in the case of removal of cancerous tumors, Da Vinci performs such an operation, he literally cuts off one cellular layer of tissue, immediately conducts a biopsy. If no cancer cells are found in this layer, then it cuts off two more layers for reliability. And thus, as it were, the invasiveness of the intervention is reduced. That is, where a surgeon would cut off a fair amount of healthy tissue, Da Vinci will perform the operation with surgical precision. In general, initially this solution was developed for the military. It was believed that it would be possible to use it in field hospitals. But the military recognized this decision as impractical, so it went to the civilian sphere. In 2012 , 200 thousand operations using Da Vinci were performed in the world. The number is only growing. Russia is lagging behind in the number of such robots, but, nevertheless, there are already about a hundred of them in Russia. There are several thousand around the world , probably closer to 10 thousand.

About the marages I mentioned, the most famous of them, that is, these are unions of Data Science companies with pharmaceutical giants, here are 4, probably the most famous: Google DeepMind has stated its claims on the topic of finding new drugs. They are now a joint project with the British they open it, we don't know the details, but it was stated that the work this is being conducted. Then Pfizer and IBM signed a joint project, also dedicated to the search for drugs. Sanofi and GENZYME and Recursion Pharmaceuticals with GSK and Exscientia and Evotec with Celgene. Here is a set of unions. Let's see, soon we should see some first results of this direction. So far, the expectations are very rosy, the volume of investments is large in this area.

Well, about bionic prostheses, which I have already mentioned a little bit, this is just a cochlear implant, you can see how it all looks. The transmitting coil, the receiver and, as a matter of fact, we start the signal right into the snail. Here's the baby. My friends have a child with such a cochlear implant, and the operation was probably done about 6 years ago, that is, then it was already more or less common. Now it is no longer considered high-tech. These are the pictures of these horrors by William Dobell. This is Jens Naumann just shown here with his magical device. In fact, the history of Dobell's prostheses specifically did not end very well, because Dobell had the imprudence to die quite unexpectedly (it is not very often "expected" at all). And like all sorts of people who are very much ahead of themselves, he did not bother himself very much with a good documentation of what he did. And, generally speaking, what was left after him, it was, of course, not very usable. Of course, other laboratories have caught up and surpassed Dobell's successes to date, but specifically for those 16 patients who had Dobell devices installed in 2002, they have all lost their sight for the second time today, because, of course, the technology is crude. That is, there, firstly, in a number of cases, infectious complications occurred, specifically in Jansa Naumann, due to infectious complications, were forced to remove the interface, this is where it is after the operation, after removal. And in addition there is a problem there tissue scarring, myelination, that is, over time, such systems begin to transmit a signal more and more poorly. But the fact is that, of course, this is already the last century, because optogenetic solutions have now replaced such interfaces based on electrodes implanted in the brain . Optogenetics, in short, is an approach in which we modify neurons with the help of a viral vector, growing photorecepts based on the protein channelorodopsin-2 (ChR2) on them. And then you can shine light on this part of the brain that you have transformed in this way, and, accordingly, the neurons will excitement will arise, and this signal will be transmitted further through the neural network. With the help of such technology , for example, a number of projects with radio-controlled animals have now been done. A radio-controlled dragonfly, for example, at the beginning of last year such a very beautiful projection was shown, you can just search for Cyborg on YouTube Dragonfly – and there, accordingly, there will be a lot of videos on this topic, how it all works. There, the dragonfly not only carries this micro device, which transmits signals to her brain that make her fly in different directions, she also carries a micro-GPS sensor with full-fledged navigation, that is, in general, such a device is already quite adult. And so, in principle, there are many projects on this topic, and last year the use of optogenetics on humans was approved for the first time. A small company received this permission from the FDA, and it was immediately bought by the pharmaceutical giant Allergan, so far in order to restore vision, of course. That is, where the layers are damaged, photoreceptors are lost, it is assumed that microinjections of these viral vectors will be carried out, and new photoreceptors will be created. I think there is more to come, and we will see, of course, a lot of interesting things here. Moreover, nature shows us in general a lot of very interesting examples that such technologies are quite viable. The example of two twin sisters is very indicative Krista and Tatiana Hogan. These are craniopagus girls. Craniopages are Siamese twins who fuse their heads in the womb. And specifically in the case of Krista and Tatiana Hogan, they grew together not only with their heads, but also with their brains. And between the brains of these girls , such a structure was formed, which was called the thalamic bridge. And it has very interesting properties. Thanks to her, girls can exchange images with each other, that is, being separated by an opaque screen, they are able to transmit information about the objects they observe, while they have completely independent personalities, that is, each of them has their own tastes, their individual preferences. At the same time, they are able to communicate effectively using this thalamic bridge. And this example shows us that sooner or later, when the level of technology development allows us, thanks to neuroplasticity, we will most likely be able to create such systems and not only to restore vision, but also for much more fun things.

Recognition of medical images. Here there is a model called U-Net. This is, strictly speaking, the heir of architecture RezNet for pattern recognition. With the help of this architecture, a lot of tasks have been solved for the recognition of medical images in various fields, for example, in cytology – coloring of cells, cell cultures. A blue color like this is what the model recognizes the boundaries of the cell, and yellow is a manual marking, in fact, what it was. And, accordingly, they are used in a very, very large number of directions today. Periodic competitions on recognition of medical images are held. Probably the most famous is the Grand Challenge for Computer-Automated Detection... ah, this is the ISBI 2015 big conference, and here are some "sub-challenges" in it, as it were. Here caries was recognized. And the Cell Tracking Challenge is here specifically U-Net won these two challenges. Cell Tracking is just the recognition of cells in cell cultures. But in fact , in this challenge, it has a bunch of all sorts of looking for tumors on X-rays and so on. Moreover, as if the accuracy of solving these problems is not inferior to people today. That is, in the sense that we are able to recognize objects not only there cats, dogs effectively, but also such medical objects.

Analysis of data from mobile devices. There is such a very interesting funny task here, which is almost educational, when, according to the data from the accelerometer of the phone, you have to recognize what a person is doing at the moment when such and such data from the accelerometer is recorded. That is, sitting, standing, walking, running, climbing a ladder. Accordingly, there are public datasets with this information already marked up, you can yourself download them, practice making your own neural network that will recognize movements. But in fact, it's, in general, probably a little boring. But the work that I and my team participated in last year is an attempt to predict the age of a person to begin with. That is, we take information from the sensors of the wearable device and try to understand how old this person is. The first model we made, it predicts the calendar age. The array that was the basis for it is an array called NHANES. This is a public array NHANES state, large-a large array of people's health conditions, and among other things there are 10 thousand people, for each of whom there are 10,080 samples from the phone's accelerometer with a frequency of one minute. That is, the total amount of activity per minute is integrated there, and this is one minute sample. That is, you take 10,080 samples of a person and try to predict his calendar age. It turns out that in general , the Pearson correlation coefficient between the model forecast and the real calendar age, it is more than 0.8, there is somewhere 0.82.

At the same time, it is interesting that the age that the model predicts is actually a calendar age, not a biological one – it was such a small disappointment. We thought that maybe for free it would be possible to predict the probability of death in fact. That is, to take through this age, through the Gompertz curve, to recalculate the probability of death and, opa-opa, we have a ready-made diagnostic tool. But no. Apparently, the problem here is that we have a lot of motor patterns that depend on our calendar age. That is, we go to school when we are 7 years old according to the calendar, and not when our probability of death becomes the same as the average for a seven-year-old person. The same thing happens with college, with an institute, with a pension and many other things .

That is, our motor patterns are tied to our oscillatory age. But we did not lose our heads and tried to make a model that predicts exactly the probability of a person's death, because in fact, this array contains information about whether those people died/did not die, and when they died. Well, and, accordingly, as if solving the so–called censorship problem there (from the English censoring - approx. XX2 CENTURY), we built a model that predicts the probability of a person's death in the year following the date of this sample taken. And her accuracy is certainly more modest, that is, she has a Pearson coefficient somewhere in the region of 0.16. But, nevertheless, this is a significant value. It explains some of the variance , and, in general, we are now working with a more detailed array from UK Biobank, where data from sensors is taken, and a sample is counted more than once a minute, and with a frequency of 100 hertz, the signal is captured. And the array itself is still much larger, that is, there are hundreds of thousands of patients in it. And I think that this model will still be able to clarify quite well. And, accordingly, please, in principle, we will have a working tool that, in case of anything, will at least light a red light bulb for you and advise you to consult a doctor. That is, if your probability of death exceeds a significant threshold there compared to the probability that corresponds to your age.

A little bit about interfaces, a few additional words about what can be done in medical interfaces using artificial intelligence systems. I have already talked about speech interfaces, that is, speech recognition and synthesis. In addition, in addition to recognizing and synthesizing speech, you also need to understand the meaning of what was said, that is, these are so-called intonation models that, after hearing a certain phrase from a person, should understand what he meant. Such models, which are quite effective at the moment, have been created, at least this stack is enough to already build a more or less meaningful dialogue in accordance with a certain communication script.

Image recognition, biometrics. Biometrics, of course, yes, you naturally have medical data, all medical interactions they belong to the specially protected categories of personal data processing, because medical information is, of course, a sealed secret in accordance with modern legislation. Accordingly, biometrics is in order for you to determine whether or not this is the person you are communicating with when interacting remotely.

Personalization of systems. That is, we can adjust the communication style of a virtual consultant for each person. It is important here that here, most likely, there will be a fairly big breakthrough in the coming years, now I will tell you why. Chatbots are already actively entering our lives today, they are text interaction tools with which we can effectively solve a number of tasks. Actually , image recognition is from the point of view of diagnostics during interaction.

I promised to tell you why there will be progress in this area. The fact is that I briefly mentioned that large companies are now engaged in a race to create specialized hardware for training neural networks. And in fact, some of these devices have already appeared, but it is not yet available in the user segment. For example, Google has created a piece of hardware like Tensor Processing Unit, the second version, they are still available in cloud computing, but this is a specialized processor for training the most common neural networks that we use. It was on the basis of such hardware that the recently sensational AlphaZero project was made, this is the heir of AlphaGo, which beat AlphaGo itself there and beat the world's strongest chess program Stockfish and the world's strongest Shogi program – this is Japanese chess. In many ways, the success of this project was based on the fact that specialized very fast hardware was used. That is, one such TPU 2.0 board is 280 teraflops of performance. And, of course, this is a monstrous computing power, also much more energy efficient than what we have used so far .

Other companies have such projects as well. For example, Intel has disbanded its most trump team, which is called the Core Team, which created the processor architecture at one time Core 2 Duo, and based on it created two teams to create specialized chips for neural networks. One of these projects is called Nervana, and, most likely, we should soon see the fruits of their labor. Plus, this is IBM with its TrueNorth, plus information has leaked that there are relevant works in the union Facebook and AMD, although they are not officially confirmed, but the leak was published in CNBC, that is, such a more or less reliable source of information.

In general, there is a big race going on, and this means that in a few years in our user segment , specialized hardware for neural networks will appear on all home devices, portable and mobile , which will allow not only to calculate them, because you can, in principle, quickly calculate a trained neural network on a modern mobile on the phone, unless, of course, it is of some exorbitant size. But you can also train neural networks directly on the user device. This means that you will be able to solve a number of intellectual tasks on the client side. And in this sense, one of the obvious directions is personalization. Personalization in the sense that it is a system that will learn to interact most effectively with you, will understand your hints, your voice, your gestures, your facial movements, and so on and so forth. But this is one of such obvious by-products that this kind of hardware will appear.

Now thank you for your attention. Two links: this is a link to my website, which has, firstly, all sorts of video recordings of my various lectures, and secondly, there are sources for this lecture, that is, there are links where you can go and read more about what I was talking about, well, and a lot any funny nonsense and my contacts that can be used to contact me and ask me any question that I will try to answer. Here, this is the first link. All photographed, who wanted to? And the second one is a questionnaire A scientific environment that we ask you to fill out because we are interested in something.

Portal "Eternal youth" http://vechnayamolodost.ru