Brain and Computer (1)
Brain-computer interface, reality and fantasy. Part 1
Mikhail Lebedev, Professor of the Center for Neurobiology and Neurorehabilitation at Skoltech, Scientific Director of the Center for Bioelectric Interfaces at the Institute of Cognitive Neuroscience at the Higher School of Economics answers questions from Marina Astvatsaturyan, correspondent of Echo of Moscow.
M. Astvatsaturyan ― Hello, dear listeners and viewers! In today's issue of the series "Conversations for Life", we will talk about the brain-computer interface. About how to connect to the brain, what is real, what is still fiction, if there is one. I welcome to our studio a neurophysiologist, professor of the Center for Neurobiology and Neurorehabilitation of Skoltech, scientific director of the Center for Bioelectric Interfaces of the Institute of Cognitive Neuroscience of the National Research University Higher School of Economics Mikhail Lebedev. Hello, Mikhail Albertovich.
Mikhail Lebedev ― Hello, thank you for the invitation.
M. Astvatsaturyan ― Before we start talking to you quite seriously, I want to ask a question that arose from quite common, but in fact — replicated ideas, and the question is: is the brain a computer or not?
M. Lebedev ― The Brain is…
M. Astvatsaturyan ― The human brain.
M. Lebedev ― The human brain is a computer. But here it should be taken into account that all our ideas about the brain — they are always compared with our current technological development. That is, when some electronic circuits first appeared, they were popular feedback circuits, and then they said: "The brain is a feedback system." When computers appeared, they began to say: "The brain is a computer." Artificial neural networks are developing now, it has become more fashionable to say that "The brain is a neural network." So, in principle, yes, the brain is a computer in the sense that there are elements in it, they are interconnected, you can see how they work, we don't fully know how it happens yet. But, undoubtedly, the work of the brain is built roughly like certain electronic circuits, or a computer.
M. Astvatsaturyan ― But a computer is not a neural network. That is, the equal sign is so conditional after all.
M. Lebedev ― Well, of course, if we compare it literally, then the brain is very different from a computer, from those computers that we use, and by the way information is encoded, by the way information is processed. But, on the other hand, there is no such law for computers to be the way they are now. Neuromorphic computers are being created that copy how the brain works.
M. Astvatsaturyan ― When the concept of the brain-computer interface arose, what is behind these words, what is it?
M. Lebedev ― Well, generally speaking, science fiction writers have been thinking about connecting the brain to a computer or to some external device for a long time. We all know this topic, parapsychology, right? When you can move some objects with your mind. So such ideas were in the air. But probably the very first demonstration of the brain-computer interface occurred in '63, when Gray Walter demonstrated that he could connect the brains of his patients with electrodes in the motor cortex with a device such as a slide projector. He asked them to press a button and switch slides. At the same time, such a readiness potential developed in the brain — and it was strong enough — that Gray Walter was able to turn off the button and connect the readiness potential to the slide projector and the subjects switched these slides.
M. Astvatsaturyan ― Without a button.
M. Lebedev ― Without a button, yes. Moreover, according to eyewitnesses, because it was not published as a solid scientific article. The subjects were surprised how this device guesses their desire to press the button before they pressed this button.
M. Astvatsaturyan ― We will talk about this advance, but a little later.
M. Lebedev ― Well, yes, yes.
M. Astvatsaturyan ― First I want to explain the basic things, concepts, that is, so that you explain. How is brain activity recorded? Are there algorithms for decoding signals from the brain, what does this allow us to find out? Here is such a bunch of questions around what exactly makes it possible to find out the decoding of information obtained through implantation of electrodes in the brain. It's obvious that brain activity is recorded using electrodes, right? So far, at this stage.
M. Lebedev ― Yes, yes, yes.
M. Astvatsaturyan ― Here, please, here.
M. Lebedev ― Well, it should be understood here that the brain is not created for the convenience of researchers so that they can fix potentials. For example, here are modern methods of deep learning, they can recognize images. If you show them a picture, they will recognize what is in the picture. But why is that? Because there is obviously something in the picture that can be recognized. The story with the brain is somewhat different, because there are certain elements there, they work, they use both an electrical method of transmitting signals and a chemical one. And at the same time, we can somehow try to connect, but this is not such a connection as to a cable that carries convenient information for us. What is being done? It's literally inserting an electrode that's bigger than a neuron, right? That is, some kind of thin electronic network of neurons, and suddenly such a nail is inserted, and he tries to record. That is, if we imagine that, for example, we are exploring a computer in this way, it will be very, very strange that we are getting anything at all. This happens with the brain, because the brain is kind to researchers. For example, if I move my finger, and every time some certain neurons are discharged, I can say aha, these discharges of neurons are connected with the discharges of the fingers. Then, by recording the activity of these neurons, I can decode finger movements. The neurons are discharged, so the finger is bending, or I'm thinking about bending the finger. Well, then I mentioned the electrode, you can record the electrical activity of neurons, neurons are discharged with such beautiful impulses, everyone who recorded saw that it was very beautiful. In addition, non-invasive methods can be used, for example, putting electrodes on the head, no need to drill holes in the head, and they will record such more global potentials that are obtained in the form of rhythms. There's an alpha rhythm, beta rhythm, gamma, and so on. They represent the activity of a huge number of neurons, synchronous activity, so this method is somewhat different from recording directly the discharges of neurons. In addition, you can also study the blood flow of the brain, say, some area of the brain is working - there is increased blood flow, and so on, and so on.
M. Astvatsaturyan ― Linguists, you know, they even see changes in brain activity on MRI when declension of nouns: the creative case has its own brain activity, the prepositional one has its own, people receive grants for studying this.
M. Lebedev ― Yes, MRI is a very powerful method, it is especially good because its spatial resolution allows you to look deep inside the brain, where you just can't look with some surface electrodes.
M. Astvatsaturyan ― You won't look in, yes. Well, and then what does this information give, how to decipher it?
M. Lebedev ― So, then there is a very big problem of deciphering this information, because, as I said, it is not intended for our convenience so that we can decipher it, and scientists are trying to decipher it without knowing the code.
M. Astvatsaturyan ― Still not there yet, even a hint?
M. Lebedev ― It is still not known. All methods are based on correlations, that is, say, a neuron has increased activity, if I do something, it probably means that the neuron is somehow involved in this process.
M. Astvatsaturyan ― And there should be statistical reliability, it should be many, many times, the same thing?
M. Lebedev ― Yes, yes, yes. Either many, many times, or you can record many, many neurons, which also enhances the reliability of statistics. And by the way, neural interfaces try to record as many channels as possible, and thus the decoding accuracy improves.
M. Astvatsaturyan ― Decoding accuracy, which in fact is still not decoding, but rather an approximation to decoding.
M. Lebedev ― Yes, this is a kind of correlation method, and we always remember that correlation does not mean a causal relationship. That is, it should always be remembered.
M. Astvatsaturyan ― Yes, yes, it needs to be kept in mind. So far, for today.
M. Lebedev ― Yes, yes, yes.
M. Astvatsaturyan ― Maybe in 50 years we would already be talking about something else.
M. Lebedev ― It is quite possible, yes, neuroscience is developing rapidly, so we are learning more and more.
M. Astvatsaturyan ― This is generally now such a front, the vanguard of biological research, one of the.
M. Lebedev ― And it is clear why. Because the brain is somehow connected with our consciousness, so to speak, with our soul, so that even those who…
M. Astvatsaturyan ― Oh, yes. I really want to talk about this in the second part, yes.
M. Lebedev ― Good, good.
M. Astvatsaturyan ― While talking about technology, here is a synonym for the brain—computer interface, at least in popular science literature, is the expression "connect to the brain", its authorship is attributed to Ilon Mask. And his current sensational projects, I mean the neurotechnological company Neuralink, are genetically related to your work, that is, this is part of your American history, which we will touch on separately, well, please tell us about this part, about this bundle of yours. That is, the employees of Musk, his current company Neuralink, they are employees of your former laboratory, right?
Mikhail Lebedev ― Yes, yes, yes, I have worked with some of them, although two of them no longer work with Musk because of some personal considerations.
M. Astvatsaturyan ― What is he really going to do? Does he want to produce neurocomputer interfaces?
M. Lebedev ― Yes, yes, yes.
M. Astvatsaturyan ― So it's easy to produce and sell, I understand, right?
M. Lebedev ― There are two main points here. On the one hand, Musk repeats what we already did 10 years ago, he just repeats. On the other hand, it repeats this with much better technology. That is, in a short time, by investing capital and attracting engineers, he has achieved that these interfaces really become practical: small, fully implanted, communicate with the outside world wirelessly. Recently he showed a monkey, it's not even visible that this monkey is implanted, that is, this is a significant achievement. What is Musk going to do?
M. Astvatsaturyan ― Yes.
M. Lebedev ― It seems to me that he himself has a very weak idea.
M. Astvatsaturyan ― It's just fashionable, and he wanted to get into the trend, as they say.
M. Lebedev ― Apparently so, but I ... personally I am not familiar with the Mask, so I do not know.
M. Astvatsaturyan ― So you didn't read his thoughts, did you? (laughter)
M. Lebedev ― Only indirectly, according to correlational things (laughter)…
M. Astvatsaturyan ― Only monkeys, yes.
Mikhail Lebedev ― Apparently, he really feels that there is a lot of potential for development here. Moreover, in some of his speech he was asked, what about the technologies of the future? And he didn't say space, he didn't say any electric machines, but he said: "Here's the brain — yes, and that's really the future, yes!".
M. Astvatsaturyan ― Is there anything known about any fundamentally new things that will be in this chip of his that others don't have?
M. Lebedev ― There are no fundamentally new things there, at least not yet, but everything has been done very competently…
M. Astvatsaturyan ― Maybe not constructively, but ideologically, as it were?
M. Lebedev ― No, ideologically, this is our old ideology, which consists in the fact that if you insert as many electrodes into the brain as possible that will record the activity of individual neurons, then through this method you can achieve a good decoding of, as it were, anything, any thoughts, and then, since we have such access to brain activity, then we can control neuroprostheses, we can treat people, we can even improve the brain we have.
M. Astvatsaturyan ― I heard or read something about something new in signal processing, something was so unusual in this chip.
M. Lebedev ― There is a problem with the chip, it is necessary to solve the question of how much processing to provide to the chip itself, and how much to delegate to some external devices. Because if the chip simply records the activity of neurons and transmits it in such a raw form to the outside, then this is a large expenditure of energy, large demands on the communication channel, and it does not work. Therefore, the chip has to do something by itself, and in the case of Elon Musk, it does such spike sorting, that is, it looks at the discharges of neurons, and says: this neuron here is discharged, and instead of recording the entire form of this discharge, it records the time and what a neuron. And this information is sent out. And then decoding algorithms that are outside the brain come into play, some external computer is doing this. But here, Elon Musk is not unique in this.
M. Astvatsaturyan ― So there's nothing particularly breakthrough there?
M. Lebedev ― There is nothing particularly breakthrough here, but the region itself is a breakthrough, because there is still a lot of work to do here.
M. Astvatsaturyan ― Is Musk actually a startup?
M. Lebedev ― Well, a startup, yes, yes, a startup in the sense that the company does not generate revenue, and it is unknown when they will be.
Here's about interfaces, to finish with the technical part, they are mostly invasive.
M. Astvatsaturyan ― That is, it is still necessary to immerse them in the brain tissue.
M. Lebedev ― In different ways.
M. Astvatsaturyan ― Tell us.
M. Lebedev ― There are invasive cases when electrodes are placed in the brain. But there are biocompatibility problems here. That is, the brain is not a fool, so to speak, it sees that an electrode has been inserted into it, it tries to protect itself from this electrode. What's happening? At first, due to the glial cells that surround this electrode and it is, as it were, sealed, and then everything can grow with such a dense connective tissue, after which…
M. Astvatsaturyan ― How is the scar like that, right?
M. Lebedev ― Yes, yes, yes, after which the recording quality will deteriorate. By the way, if you use this electrode to stimulate the brain, then even if it is encapsulated, it will still be suitable.
M. Astvatsaturyan ― How long will he be fit?
M. Lebedev ― For the record, here it could turn out both ways. It can work for 2 weeks, and then stop working, or it can work for many years.
M. Astvatsaturyan ― Even years!
M. Lebedev ― We recorded on monkeys, there were some monkeys with these electrodes for 8 years, the recording was quite acceptable.
M. Astvatsaturyan ― What are non-invasive interfaces?
M. Lebedev ― Non—invasive - this means that we do not penetrate into the body.
M. Astvatsaturyan ― Yes, but what do you do then?
M. Lebedev ― The most popular method is to put electrodes on the scalp, on the head, and record electrical activity. The brain is a source of electromagnetic waves, and it is possible to record them completely. Probably, this has been happening for more than 100 years, such a record, and it is quite informative, and for many purposes it is quite acceptable, for example, for rehabilitation purposes. That is, for some purposes, you do not need to make high-quality neural interfaces, but you need to monitor brain activity. Let's say we are rehabilitating some patient with neurological lesions, we need to know how his activity changes depending on the therapy we are using, and this is a very useful record.
M. Astvatsaturyan ― Well, for an experimenter-researcher, invasive ones are probably better, right, after all?
M. Lebedev ― For a researcher, of course, invasive ones are better, yes. But, on the other hand, if we want to develop some kind of practical system, then we have to think about how many people there are who will want to put electrodes inside the brain. And if we create some new methods for neuroinvasive recording that hundreds of thousands of people will need, then maybe this is the way to go.
M. Astvatsaturyan ― In the USA, you conducted unique, in many ways pioneering, experiments on monkeys and rats. What are your results, exactly yours — I have seen a lot of your publications on PubMed — have raised this area to a fundamentally new level. The area, I mean the creation of a neurointerface. That's what affected the development of the entire region in the future, if there was such a thing?
M. Lebedev ― We really tried to always do pioneering research, that is, not to repeat what had already been done before us, but to strive to do something new in this area.
M. Astvatsaturyan ― Remind me, was it at Duke University?
M. Lebedev ― It was at Duke University. Well, in principle, and in other places where I worked, too. Although there was no direct topic about neural interfaces, they always tried to do something new. This is the only way to somehow distinguish yourself on the scientific front. Our first known demonstration was when we demonstrated a neurointerface that controlled such typical movements for primates — both for humans and for many animals – as reaching out and grabbing. That is, to reach out part of the movement, the second part is to grab.
M. Astvatsaturyan ― By the power of thought, so that this desire arises somehow.
M. Lebedev ― Yes, well, it was done by the power of thought, in the sense that activity was recorded in the motor cortex, and the action was carried out by a robot. It was quite a famous work. Then we wondered if we could feel this movement. What is meant here? That we control a hand that reaches out, touches some objects, but what good is it to us if it is insensitive? But if she touches an object and can feel, then…
M. Astvatsaturyan ― How do you know? If it's hotter, she pulls back, right?
M. Lebedev ― So we put sensors on this mechanical arm, and they feel the properties of these objects. That is, some kind of shape, texture, temperature, and so on.
M. Astvatsaturyan ― How do you know about it, what did they feel?
M. Lebedev ― Sensors on the robot arm can feel a lot. By the way, this is a separate topic to cover the robot with a skin that feels both temperature and touch, and so on.
M. Astvatsaturyan ― I wanted to talk about all these bionic cases, too, but a little later.
M. Lebedev ― Good, good.
M. Astvatsaturyan ― Back to your monkeys, yes.
M. Lebedev ― This was our second demonstration, where we demonstrated such a brain–machine–brain interface, which not only controlled a virtual hand, but also this virtual hand sent signals back, telling what it feels. That is, she sent such artificial tactile sensations. Then we made some progress on such an interface, which is for bimanual tasks. That is, in fact, although the experimenter is very fond of tasks where only one hand or only one finger is involved, in fact we do everything with two hands, and with one hand we do very sharply. And we made an interface where the monkey controlled two virtual hands, each moving towards its goal. And we have shown that it is really possible to achieve through the neurointerface that he controlled with two hands, independently. That is, the hands did not move in such a pathological way, say two hands in one direction, since there are such pathologies, but really independently. Well, there was also an interesting point that in order to achieve this independence, it was not necessary to record in different places of the brain, we recorded from the same neurons. This ensemble of neurons provided independent control. Then we said, well, we've done enough with the hands, let's work with the legs. And we made an interface where a monkey walked on a treadmill, and we decoded walking. That is, we recorded the activity of the brain, and decoded the walking movements so that, just by looking at how the neurons of the brain are discharged, we could tell what walking movements the monkey is doing. But this was not enough, it was necessary to demonstrate why it was necessary at all. We connected our monkey, who was in America, to a robot, to such a humanoid, who was in Japan, and he walks to the beat of the monkey.
And there was even a demonstration where we stopped the treadmill, and the monkey had an image of a robot in front of him on the screen, she continued to control it as if standing, as if controlling the robot.
M. Astvatsaturyan ― To build (laughter).
M. Lebedev ― Yes, yes, to build this robot (laughter). And further from this it follows that, generally speaking, a person whose legs are paralyzed could control an exoskeleton that is attached to the legs. And we did such demonstrations, it was also in Brazilian studies, and we also worked with a team at the Higher School of Economics, as well as with Alexey Osadchim, at the Higher School of Economics. And there we showed that due to the electroencephalogram, the walking movements of the robot can be triggered, which is necessary, generally speaking, for paralyzed people, because they need to make such Hebbian plasticity. The exoskeleton takes a step, a discharge of receptors occurs, and it rushes to the brain through some remaining fibers after a spinal cord injury, and there is a connection of synchronous brain activity, and this sensory influx, due to this, healing will begin.
M. Astvatsaturyan ― Well, that is, these are the actual artificial sensations…
M. Lebedev ― Yes, yes, yes, this is an artificial sensation, plus synchronization with the activity of the brain, and due to this, neurorehabilitation occurs.
M. Astvatsaturyan ― But since we are talking about life, I have such a question ... You know, there was a wonderful physiologist Ivan Nikolaevich Pigarev, who recently tragically died, he worked with cats. They ran around his laboratory with such constructions on their heads, with electrodes. He loved them very much, and he spoke of them as full-fledged employees. "Cats who work in our laboratory," said Ivan Nikolaevich. What is the researcher's relationship with monkeys? Let's say you went on vacation for a month, right? From Duke University, for example.
M. Lebedev ― Yes.
M. Astvatsaturyan ― Here you are back, how are these relations developing? After all, monkeys are quite highly developed creatures. Were they bored, were they offended, or did they not care? Or they just came to work, that's how your colleagues are, and they generally don't care how you rested.
M. Lebedev ― Every researcher has a different relationship with monkeys. That is, here I observed a large spectrum, up to the point that, indeed, the researcher begins to be friends with his monkey…
M. Astvatsaturyan ― Well, monkeys are probably different too, right?
M. Lebedev ― They have a trusting relationship. Monkeys are also completely different personalities. And there are some kind of scientists who don't even look at the monkey when they record it. Here he needs to look at the oscilloscope, where the discharges of neurons are, this interests him. So there is no certainty here.
M. Astvatsaturyan ― How are you?
Mikhail Lebedev ― I'm probably somewhere between these two facets.
M. Astvatsaturyan ― That is, sitting Skolkovo, you don't miss them. Well, that's the main question. Well, I won't tell the monkeys anything (laughter)
M. Lebedev ― Well, I've had so many monkeys, so I've already…
M. Astvatsaturyan ― And, of course, you can't mention everyone.
M. Lebedev ― You can't remember all of them, yes.
M. Astvatsaturyan ― Good for you. I would also like to ask you, concluding this part, we have a few minutes to tell you about the direction of your work at the Center for Neurobiology and Neurorehabilitation of Skoltech. He's young enough, this Center, right? Since 2018. But, on the other hand, already established. Judging by the name and from what I saw on the site, it is of a medical orientation. And judging by the partners: you have a Burdenko clinic in partners there, a Neurology Center. What will happen there, what is being done there, will be done?
M. Lebedev ― Yes, the main direction is always medical, because, of course, we want to do pure science, but it somehow needs to be justified, and really bring practical benefits. And in neuroscience, we can really bring practical benefits to people. People live longer, and neurological diseases are becoming more and more common.
M. Astvatsaturyan ― Well, yes, with the aging of the population, of course.
M. Lebedev ― That is, we do not want a person to live for a long time, but at the same time lose some brain functions, so we need to monitor. Monitor the health of the brain, recognize, diagnose some diseases, anticipate them. If they happen, we either rehabilitate them, that is, we use our own brain capabilities so that the brain repairs itself, or we try to compensate, replace, that is, by some means we try to make up for the lost properties of the brain. But we need such a multidisciplinary approach here, and we are trying to make sure that all areas are represented in our center — from molecules to medicine and brain systems, plus mathematics, plus robotics, so we will attract even more specialists.
M. Astvatsaturyan ― To date, you have established cooperation with the Burdenko Institute of Neurosurgery and the Center for Neurology. Do you go there to see patients, or do you consult patients in some other way, or doctors who observe these patients? Is this work already underway?
Mikhail Lebedev ― What we are doing now is that we are deploying work in several medical centers, in fact, all over the country from Vladivostok to St. Petersburg, and Kaliningrad, and Samara, and Kazan, and Moscow. And we are working on such a simulator for people who have suffered a stroke or spinal cord injury. In short, we remove the electroencephalogram in a non-invasive way, translate it into motor commands, that is, the desire to make a movement, and at the same time, since these people are paralyzed, their hand is moved by such a specially designed robot and directs to the target, and to this we also add additional means such as spinal cord stimulation, because it is known that spinal cord stimulation is a powerful tool.
M. Astvatsaturyan is the engine of activity, yes.
M. Lebedev ― Yes, to modulate motor activity, and to direct these plastic mechanisms of the brain in the right direction so that the brain corrects itself after such injuries.
M. Astvatsaturyan ― Are you not faced with the conservatism of clinicians, doctors are willing to cooperate with this?
Mikhail Lebedev ― On the contrary, I would say that we are faced with a very great desire to include all these methods. I have talked to many doctors, and they say that standard methods of treating patients after a stroke are not enough, let's introduce neurointerfaces, because this is a new direction and they really significantly improve the treatment of such patients.
M. Astvatsaturyan ― Let me remind you that Mikhail Lebedev, a neurophysiologist, professor of the Center for Neurobiology and Neurorehabilitation of Skoltech, was a guest in our "Conversations for Life" cycle. We will continue our conversation in the next part.
Portal "Eternal youth" http://vechnayamolodost.ru