Cognitive intelligent control of a robotic prosthesis arm. Part 1

Cognitive intelligent control of a robotic prosthesis arm. Part 1

Sergey V. Ulyanov
Doctor of Physical and Mathematical Sciences, Dubna State University, Institute of the System Analysis and Management, Professor, 19, Universitetskaya ul., Dubna, Moscow region, 141980, Russia, tel.: +7(49621)66010, This email address is being protected from spambots. You need JavaScript enabled to view it. 

Andrey G. Reshetnikov
PhD in Technical Sciences, Dubna State University, Institute of the System Analysis and Management, Assistant Professor, 19, Universitetskaya ul., Dubna, Moscow region, 141980, Russia, tel.: +7(49621)66010, This email address is being protected from spambots. You need JavaScript enabled to view it. 

Alexey V. Nemchaninov *
Dubna State University, Institute of the System Analysis and Management, Second Year Master Student, 19, Universitetskaya ul., Dubna, Moscow region, 141980, Russia, tel.: +7(962)249-65-76, This email address is being protected from spambots. You need JavaScript enabled to view it. 

Received 06 September 2019 

Abstract
The principles of interface «brain-computer-device» are considered. Neurointerfaces of this type allow to restore and expand the capabilities of a person with physical limitations (for example, disability with loss of limbs) or mental disorders in various activities (for example, autistic children or patients with impaired mental activity - dementia). Cognitive interfaces provide the ability to communicate, evaluate emotions, transfer and control devices with mental commands. The process of developing a cognitive intelligent simulator is discussed, the application of which provides developers with the ability to control robotic devices at the lower (executive) level, using the so-called «mental commands», and at the upper level - the intelligent level, to develop cognitive intelligent control technologies with the possibility of application in applied tasks. A description of the design stage and results of the development of the conceptual structure of a robotic prosthesis are given. As a result, a prototype of prosthesis made with use of 3D printer, conceptual structure of interaction of neural interface «brain-computer-device» as well as a settable basis for intelligent computing and for software are presented. The application of soft computing technology (the first step of IT) allows to extract knowledge directly from the physical signal of the electroencephalogram, as well as to form knowledge-based intelligent robust control of the lower performing level, taking into account the assessment of the patient's emotional state. The possibilities of applying quantum soft computing technologies (the second step of IT) in the processes of robust filtering of electroencephalogram signals for the formation of mental commands discussed. A description of a hierarchical intelligent control system based on SCOptKBTM (knowledge base optimizer) using soft and quantum computing technologies is presented.

Key words
Robotic prosthetic arm, intelligent cognitive computing, «brain-computer-device» neurointerface, mental commands, quantum soft computing, cognitive controller.

Acknowledgements
This work was carried out with financial, material and technical support of INESIS LLC (EFKO Group of Companies), and in association with Feng Maria (Civil Engineering and Engineering Mechanics, Columbia University).

DOI
https://doi.org/10.31776/RTCJ.7407 

Bibliographic description
Ulyanov, S., Reshetnikov, A. and Nemchaninov, A. (2019). Cognitive intelligent control of a robotic prosthesis arm. Part 1. Robotics and Technical Cybernetics, 7(4), pp.306-317.

UDC identifier:
004.415.2, 004.588

References

  1. Miranda, R., Casebeer, W., Hein, A., Judy, J., Krotkov, E., Laabs, T., Manzo, J., Pankratz, K., Pratt, G., Sanchez, J., Weber, D., Wheeler, T. and Ling, G. (2015). DARPA-funded efforts in the development of novel brain–computer interface technologies. Journal of Neuroscience Methods, 244, pp.52-67.
  2. Sberbank Robotic Lab. (2019). An analytical review of the global robotics market. p.272.
  3. Yang, G., Bellingham, J., Dupont, P., Fischer, P., Floridi, L., Full, R., Jacobstein, N., Kumar, V., McNutt, M., Merrifield, R., Nelson, B., Scassellati, B., Taddeo, M., Taylor, R., Veloso, M., Wang, Z. and Wood, R. (2018). The grand challenges ofScience Robotics. Science Robotics, 3(14), p.eaar7650.
  4. Sberbank Robotic Lab. (2018). An analytical review of the global robotics market. p.79.
  5. Sberbank (2019). Artificial Intelligence: Approaches to Formation. AI development strategies in the Russian Federation.
  6. Amirova, É., Efimov, V., Kuzhekin, A., Lunina, N., Slepchenko, A., Ul'yanov, S., Khanukaev, A., Shakhnazarov, M., Shakhnazarova, I. and Shishkin, B. (1991). Expert system for selecting lower-extremity (thigh) prostheses and diagnosis of the quality of artificial replacement. Part I. Biomedical Engineering, 25(3), pp.118-126.
  7. Amirova, É., Efimov, V., Kuzhekin, A., Lunina, N., Slepchenko, A., Ul'yanov, S., Khanukaev, A., Shakhnazarov, M., Shakhnazarova, I. and Shiskhin, B. (1991). An expert system for selection of lower extremity (thigh) prosthesis and evaluation of prosthetic quality (part II). Biomedical Engineering, 25(6), pp.270-281.
  8. Lupina, I., Slepchenko, A., Ulyanov, S. and et al (1991). Gibridnaya ekspertnaya sistema s glubinnym predstavleniyem znaniy dlya proyektirovaniya i diagnostiki biotekhnicheskikh izdeliy [Hybrid expert system with an in-depth representation of knowledge for the design and diagnosis of biotechnological products]. Izv. USSR Academy of Sciences. Ser. Technical cybernetics, 5, pp.152-175. (in Russian).
  9. Kaplan, A., Shishkin, S., Ganin, I. and et al (n.d.). The prospects of the P300-based brain computer interface in game control. In: IEEE Transactions on Computational Intelligence and AI in Games, 2013.
  10. Shishkin, S., Ganin, I. and Kaplan, A. (2011). Event-related potentials in a moving matrix modification of the P300 brain–computer interface paradigm. Neuroscience Letters, 496(2), pp.95-99.
  11. Hochberg, L., Serruya, M., Friehs, G., Mukand, J., Saleh, M., Caplan, A., Branner, A., Chen, D., Penn, R. and Donoghue, J. (2006). Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature, 442(7099), pp.164-171.
  12. Höller, Y., Bergmann, J., Kronbichler, M., Crone, J., Schmid, E., Thomschewski, A., Butz, K., Schütze, V., Höller, P. and Trinka, E. (2013). Real movement vs. motor imagery in healthy subjects. International Journal of Psychophysiology, 87(1), pp.35-41.
  13. Salvaris, M., Cinel, C., Citi, L. and Poli, R. (2012). Novel Protocols for P300-Based Brain–Computer Interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20(1), pp.8-17.
  14. Musallam, S. (2004). Cognitive Control Signals for Neural Prosthetics. Science, 305(5681), pp.258-262.
  15. Ivanitsky, G. (2008). Opredeleniye kharaktera myslitel'noy deyatel'nosti po ritmicheskomu risunku EEG [Determining the nature of mental activity according to the rhythmic pattern of the EEG]. [online] Rfbr.ru. Available at: https://www.rfbr.ru/rffi/portal/project_search/o_53385 [Accessed 11 Nov. 2019]. (in Russian).
  16. Sunny, T., Aparna, T., Neethu, P., Venkateswaran, J., Vishnupriya, V. and Vyas, P. (2016). Robotic Arm with Brain – Computer Interfacing. Procedia Technology, 24, pp.1089-1096.
  17. Gandhi, V. (2015). Brain-computer interfacing for assistive robotics. New York: Academic Press.
  18. Meng Jing (2019). BrainCo CEO says 'mind-reading' tech is about focus, not surveillance. [online] South China Morning Post. Available at: https://www.scmp.com/tech/innovation/article/3008439/brainco-ceo-says-his-mind-reading-tech-here-improve-concentration [Accessed 11 Nov. 2019].
  19. Yanagisawa, K., Tsunashima, H. and Sakatani, K. (2012). Brain-Computer Interface Using Near-Infrared Spectroscopy for Rehabilitation. Infrared Spectroscopy - Life and Biomedical Sciences.
  20. Vaughan, T., Mcfarland, D., Schalk, G., Sarnacki, W., Krusienski, D., Sellers, E. and Wolpaw, J. (2006). The Wadsworth BCI Research and Development Program: At Home With BCI. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 14(2), pp.229-233.
  21. Ulyanov, S., Shevchenko, A., Mamaeva, A. and et al (n.d.). Soft Computing Optimizer as Deep Machine Learning in Intelligent Cognitive Estimation of Human Being Emotion for Robotic Control. Robots and Engineering Cybernetics (in press).
  22. Rao, R. (2019). Towards neural co-processors for the brain: combining decoding and encoding in brain–computer interfaces. Current Opinion in Neurobiology, 55, pp.142-151.
  23. Millán, J., Ferrez, P. and Buttfield, A. (n.d.). Non Invasive Brain-Machine Interfaces, IDIAP Research Institute.
  24. Maksimenko, V., Kurkin, S., Pitsik, E., Musatov, V., Runnova, A., Efremova, T., Hramov, A. and Pisarchik, A. (2018). Artificial Neural Network Classification of Motor-Related EEG: An Increase in Classification Accuracy by Reducing Signal Complexity. Complexity, 2018, pp.1-10.
  25. Ulyanov, S., Reshetnikov, A., Mamaeva, A. and et al (2010). Gibridnye kognitivnye sistemy ypravleniya na primere upravleniya transportnym sredstvom [Hybrid cognitive control systems on the example of driving]. Journal of Systems Analysis in Science and Education, 3.
  26. Ulyanov, S. and Reshetnikov, A. (2016). Bazis kognitivnogo komp'yuternogo obucheniya robototekhnike. Intellektual'nyy trenazher formirovaniya aktivnykh znaniy [Basis of cognitive computer training in robotics. Intellectual simulator for the formation of active knowledge]. Journal of Systems Analysis in Science and Education, 4. (in Russian)
  27. Langevin, G. (n.d.). InMoov – open-source 3D printed life-size robot. [online] Inmoov.fr. Available at: http://inmoov.fr [Accessed 11 Nov. 2019].
  28. Thingiverse.com. (2012). Hand robot InMoov by Gael_Langevin. [online] Available at: https://www.thingiverse.com/thing:17773 [Accessed 11 Nov. 2019].
  29. EMOTIV. (n.d.). Emotiv Epoc, Neirocomputerniy interface [neurocomputer interface]. [online] Available at: https://www.emotiv.com/ [Accessed 11 Nov. 2019].
  30. Arduino.ru. (n.d.). Arduino, electronic device design tool. [online] Available at: http://arduino.ru/ [Accessed 11 Nov. 2019].
  31. Petrov, B., Ulanov, G., Ulyanov, S. and et al (1978). Teoriya modeley v protsessakh upravleniya: informatsionnyy i termodinamicheskiy aspekty [The theory of models in control processes: information and thermodynamic aspects]. Moscow: Nauka Publ., p.224. (in Russian).
  32. Ulyanov, S. (2007). Soft computing optimizer of intelligent control system structures. 7,219,087B2.
  33. Ulyanov, S. (2006). System for soft computing simulation. 0218 A1.
  34. Sotnokov, P. (2014). Obzor metodov obrabotki signala electroencifologrammy v interfeisah mozg komputer [Overview of EEG Signal Processing Techniques at Brain Computer Interfaces]. Engineering Bulletin, Electronic Journal, 10. (in Russian).
  35. Ulyanov, S. and et al (2015). Intellektual'nyye samoorganizuyushchiyesya kognitivnyye regulyatory. CH. 2: modeli kognitivnykh interfeysov «mozg – ustroystvo» [Intelligent self-organizing cognitive regulators. Part 2: models of cognitive interfaces «brain – device»]. Journal of Systems Analysis in Science and Education, 1. (in Russian).
  36. Ulyanov, S. (2013). Intelligent self-organized robust control design based on quantum / soft computing technologies and Kansei engineering. Computer Science Journal of Moldova, 21(2(62), pp.242-279.
  37. Ulyanov, S. and Yamafuji, K. (2014). Intelligent self-organized cognitive controllers. Pt. 1: Kansei / affective engineering and quantum / soft computing technologies. System Analysis in Science and Education, 4.
  38. Ulyanov, S. (2019). Quantum Fast Algorithm Computational Intelligence PT I SW HW Smart Toolkit. Artificial Intelligence Advances, 1(1).
  39. Ulyanov, S., Reshetnikov, A. and Tyatyushkina, O. (2019). Intellektual'naya robototekhnika ch. 2: sotsio - ekonomiko - tekhnicheskaya platforma kognitivnogo obrazovatel'nogo protsessa [Intellectual robotics, part 2: socio-economic-technical platform of the cognitive educational process]. Journal of Systems Analysis in Science and Education, 4. (in Russian).
  40. Bagdasaryan, N., Reshetnikov, P., Ulyanov, S. and et al (2019). Nestandartnaya logika nauchnykh bazisov obrazovatel'nykh protsessov v skvoznykh informatsionnykh tekhnologiyakh i kognitivnykh vychisleniyakh. CH. 1: problema Bekona, analitika bol'shikh dannykh i teoriya intellektual'nykh modeley poznaniya fizicheskikh [Non-standard logic of scientific bases of educational processes in cross-cutting information technologies and cognitive computing. Part 1: Bacon's problem, big data analytics and the theory of intellectual models of cognition of physical processes]. Journal of Systems Analysis in Science and Education, 1. (in Russian).
Editorial office address: 21, Tikhoretsky pr., Saint-Petersburg, Russia, 194064, tel.: +7(812) 552-13-25 e-mail: zheleznyakov@rtc.ru