Cognitive intelligent control. Part I: System of operator emotions’ assessment with application of deep machine learning based on soft computing

Cognitive intelligent control. Part I: System of operator emotions’ assessment with application of deep machine learning based on soft computing

Sergey V. Ulyanov
Doctor of Physical and Mathematical Sciences, Dubna State University, Institute of the Sys-tem Analysis and Management, Professor, 19, Universitetskaya ul., Dubna, Moscow region, 141980, Russia; INESYS LLC (EFKO GROUP), 20, Bld. 1, Ovchinnikovskaya naberezhnaya, Moscow, 115035, Russia, tel.: +7(49621)6-40-19, This email address is being protected from spambots. You need JavaScript enabled to view it.

Andrey V. Shevchenko
Dubna State University, Institute of the System Analysis and Management, Postgradu-ate Student, 19, Universitetskaya ul., Dubna, Moscow region, 141980, Russia; INESYS LLC (EFKO GROUP), 20, Bld. 1, Ovchinnikovskaya naberezhnaya, Moscow, 115035, Russia

Alla A. Mamaeva
Dubna State University, Institute of the System Analysis and Management, Postgraduate Stu-dent, 19, Universitetskaya ul., Dubna, Moscow region, 141980, Russia; INESYS LLC (EFKO GROUP), 20, Bld. 1, Ovchinnikovskaya naberezhnaya, Moscow, 115035, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it.

Received 13 January 2020

Abstract
The article consists of two parts. The first part considers the possibilities of application of intelligent computing technologies in the form of deep machine learning. Software-algorithmic support for machine learning is based on the knowledge base optimizer on soft computing, using the SCOptKB ™ software toolkit. The knowledge base optimizer is based on three genetic algorithms, implements the optimal structure of a fuzzy neural network and plays the role of a universal approximator of the training signal with required accuracy. The application of this approach in the tasks of cognitive intelligent control with the use of a cognitive helmet as a neurointerface is considered. The purpose of this work is to show experimentally the possibility of mental states’ classifying for a human operator, to reveal objective indicators of the psychophysiological state of the investigated person using intelligent computing technologies such as soft computing. The role and necessity of application of intelligent computing for the task of describing the general psychophysiological state of a human operator is shown with examples. The developed information technologies are considered on special (difficult for diagnostic practice) examples of assessing the emotional state of children with autism spectrum disorders. The history of the creation of knowledge bases for an intelligent service robot is also presented.

Key words
Neurointerface, intelligent computing, intelligent control system, deep machine learning, emotions, autism spectrum disorders.

DOI
https://doi.org/10.31776/RTCJ.8307

Bibliographic description
Ulyanov, S., Shevchenko, A. and Mamaeva, A. , 2020. Cognitive intelligent control. Part I: System of operator emotions’ assessment with application of deep machine learning based on soft computing. Robotics and Technical Cybernetics, 8(3), pp.217-232.

UDC identifier:
004.89-047.58

References

  1. Petrov, B. et al., 1977. Informatsionno-Semanticheskie Problemy v Protsessakh Upravleniya i Organizatsii [Informational and Semantic Problems in Control and Managemant Processes]. Moscow: Nauka Publ. (In Russian).
  2. Ozer, E. and Feng, M., 2019. Structural reliability estimation with participatory sensing and mobile cyber-physical structural health monitoring systems. Applied Sciences, 9(14), p.2840.
  3. Noor, A., 2015. Potential of cognitive computing and cognitive systems. Open Engineering, 5(1).
  4. Hieida, C., Horii, T. and Nagai, T., 2018. Deep emotion: A computational model of emotion using deep neural networks. [online] arXiv.org. Available at: <http://arxiv.org/abs/1808.08447> [Accessed 18 March 2020].
  5. Rozaliev, V., 2009. Postroenie matematicheskoi modeli emotsii: Integrirovannye modeli i myagkie vychisleniya v iskusstvennom intellekte [Building a mathematical model of emotions: Integrated models and soft computing in artificial intelligence]. In: Sbornik trudov V mezhdunarodnoy nauch.-prakt. konferentsii «Integrirovannye modeli i myagkie vychisleniya v iskusstvennom intellekte», T.2 [Proceedings of V International Scientific and Practical Conference on Integrated Models and Soft Computing in Artificial Intelligence, Vol.2]. pp.950-957. (In Russian).
  6. Bazgir, O., Mohammadi, Z. and Habibi, S., 2018. Emotion recognition with machine learning using EEG signals. 25th National and 3rd International Iranian Conference on Biomedical Engineering (ICBME).
  7. Huang, H., Xie, Q., Pan, J., He, Y., Wen, Z., Yu, R. and Li, Y., 2019. An EEG-based brain computer interface for emotion recognition and its application in patients with disorder of consciousness. IEEE Transactions on Affective Computing, pp.1-1.
  8. Ul'yanov, S. et al., 2011. Intellektual'noe Robastnoe Upravlenie: Tekhnologii Myagkikh Vychislenii [Intelligent Robust Control: Soft Computing Technologies]. p.406. (In Russian).
  9. Ul'yanov, S. et al., 1997. Development of intelligent mobile robots for service use and mobile automation systems including wall climbing robots: Pt. 1. Fundamental design principles and motion models. International Journal of Intelligent Mechatronics: Desing and Production, 1(3), pp.111-143.
  10. Tanaka, T., Ohwi, J., Litvintseva, L., Yamafuji, K. and Ulyanov, S., 1996. Intelligent control of a mobile robot for service use in office buildings and its soft computing algorithms. Journal of Robotics and Mechatronics, 8(6), pp.538-554.
  11. Dawson, G. and Toth, K., 2015. Autism spectrum disorders. Developmental Psychopathology, pp.317-357.
  12. Stanton, C., Kahn Jr., P., Severson, R., Ruckert, J. and Gill, B., 2008. Robotic animals might aid in the social development of children with autism. Proceedings of the 3rd International Conference on Human Robot Interaction - HRI '08.
  13. Wei, C. et al., 2019. Could interaction with social robots facilitate joint attention of children with autism spectrum disorder? Computers in Human Behavior, pp. 98.
  14. Palestra, G., Carolis, B. and Esposito, F., 2017. Artificial Intelligence for robot-assisted treatment of autism. In: WAIAH@AI*IA.
  15. Cho, S. and Ahn, D., 2016. Socially assistive robotics in autism spectrum disorder. Hanyang Medical Reviews, 36(1), p.17.
  16. Ulyanov, S., 2007. Soft Computing Optimizer of Intelligent Control System Structures. 7,219,087B2.
  17. Rudovic, O., Lee, J., Dai, M., Schuller, B. and Picard, R., 2018. Personalized machine learning for robot perception of affect and engagement in autism therapy. Science Robotics, 3(19), p.6760.
  18. Ulyanov, S., Mamaeva, A. and Shevchenko, A., 2018. Programmnaya realizatsiya modulya obrabotki dannykh dlya kognitivno-intellektual'noi sistemy dlya detei-autistov [Software implementation of the data processing module for the cognitive-intellectual system for autistic children]. In: Sbornik dokladov XXV mezhdunarodnoi konferentsii «MATEMATIKA. KOMP''YuTER. OBRAZOVANIE» [Proceedings of the XXV International Conference on Mathematics, Computer, Education]. (In Russian).
  19. Ul'yanov, S., Mamaeva, A. and Shevchenko, A., 2016. Kognitivno-intellektual'naya sistema diagnostiki, obucheniya i adaptatsii detei-autistov [Cognitive-intellectual system of diagnostics, training and adaptation of autistic children]. Sistemnyi Analiz v Nauke i Obrazovanii [System Analysis in Science and Education], [online] 5. Available at: <http://http:/www.sanse.ru/archive/42> [Accessed 18 March 2020]. (In Russian).
  20. Ulyanov, S., Mamaeva, A. and Shevchenko, A., n.d. Kognitivno-intellektual'naya sistema diagnostiki, obucheniya i adaptatsii detei-autistov [Cognitive-intellectual system of diagnostics, training and adaptation of autistic children]. Programmnye Produkty i Sistemy [Software Products and Systems], [online] Available at: <http://swsys-web.ru/cognitive-intellectual-system-for-diagnosis-and-education-of-autistic-children-2.html> [Accessed 18 March 2020]. (In Russian).
  21. Ulyanov, S. et al., 2016. Gibridnye kognitivnye sistemy upravleniya na primere upravleniya transportnym sredstvom [Hybrid cognitive control systems on the example of vehicle control]. Sistemnyi Analiz v Nauke i Obrazovanii [System Analysis in Science and Education], [online] 2. Available at: <http://http:/www.sanse.ru/archive/40> [Accessed 18 March 2020]. (In Russian).
  22. Nikolaev, A., 1994. Spektral'nye kharakteristiki EEG na pervom etape resheniya razlichnykh prostranstvennykh zadach [Spectral characteristics of EEG at the first stage of solving various spatial problems]. Psikhologicheskii Zhurnal [Psychological Journal], 15(6), pp.100-106. (In Russian).
  23. Lapshina, T., 2006. Psikhofiziologicheskaya diagnostika emotsii cheloveka po pokazatelyam EEG [Psychophysiological diagnostics of human emotions by EEG indicators]. In: Mezhd. nauch.-prakt. konferentsiya «Razvitie nauchnogo naslediya Borisa Mikhailovicha Teplova v otechestvennoi i mirovoi nauke: sbornik» [Proceedings of Int. Scientific and Practical Conference on Development of the Scientific Heritage of Boris Mikhailovich Teplov in Domestic and World Science]. pp.160-165. (In Russian).
  24. Fretska, E. et al.,1999. Loss of control and negative emotions: a cortical slow potential topography study. International Journal of Psychophysiology, 33, pp.127-141.
  25. Ulyanov, S., Reshetnikov, A. and Mamaeva, A., 2017. Gibridnye kognitivnye nechetkie sistemy upravleniya avtonomnym robotom na osnove neirointerfeisa i tekhnologii myagkikh vychislenii [Hybrid cognitive fuzzy control systems for an autonomous robot based on a neurointerface and soft computing technology]. Programmnye Produkty i Sistemy [Software Products and Systems], 30(3), pp.420-424. (In Russian).
  26. Ulyanov, S. and Yamafuji, K., n.d. Fuzzy intelligent emotion and instinct control of a robotic unicycle. Proceedings of 4th IEEE International Workshop on Advanced Motion Control - AMC '96 - MIE.
  27. Ulyanov, S., Watanabe, S. and Yamafuji, K., 1996. A new physical measure for mechanical controllability of a robotic unicycle on basis of intuition, instinct and emotion computing. In: Proc. of 2nd Intern. Conf. on Application of Fuzzy Systems and Soft Computing. pp.78-92.
  28. Ulyanov, V., Ohkura, T., Yamafuji, K. and Ulyanov, S., n.d. Intelligent control of an extension-cableless robotic unicycle: A study of mechanical controllabilty via minimum entropy. Progress in System and Robot Analysis and Control Design, pp.559-570.
  29. Hagiwara, T. et al., 2003. An Application of a Smart Control Suspension System for a Passenger Car Based on Soft Computing. Yamaha Motor Technical Report.
  30. Ulyanov, S., 2004. Intelligent Mechatronic Control Suspension System Based On Quantum Soft Computing. WO 2004/012139.
  31. Ulyanov, S., 2006. System for Soft Computing Simulation. 2006,0218 A1.
  32. Gandhi, V., Prasad, G., Coyle, D., Behera, L. and McGinnity, T., 2014. Quantum neural network-based EEG filtering for a brain–computer interface. IEEE Transactions on Neural Networks and Learning Systems, 25(2), pp.278-288.
  33. Ulyanov, S., Feng, M., Ulyanov, V., Yamafuji, K., Fukuda, T. and Arai, F., 1998. Stochastic analysis of time-variant nonlinear dynamic systems. Part 1: The Fokker-Planck-Kolmogorov equation approach in stochastic mechanics. Probabilistic Engineering Mechanics, 13(3), pp.183-203.
  34. Ulyanov, S., 2013. Intelligent self-organized robust control design based on quantum / Soft computing technologies and Kansei engineering. Computer Science Journal of Moldova, 21(2), pp.242-279.
  35. Ulyanov, S. and Yamafuji, K., 2014. Intelligent self-organized cognitive controllers. Pt. 1: Kansei / Affective engineering and quantum / soft computing technologies. System Analysis in Science and Education, [online] 4. Available at: <http://available at: http:/www.sanse.ru/archive/48> [Accessed 18 March 2020].
  36. Jonell, P. and et al., 2017. Machine Learning and Social Robotics for Detecting Early Signs of Dementia. [online] arXiv. Available at: <https://arxiv.org/pdf/1709.01613.pdf> [Accessed 18 March 2020].
  37. Ulyanov, S., 2019. Quantum fast algorithm computational intelligence PT I SW HW Smart toolkit. Artificial Intelligence Advances, 1(1).
Editorial office address: 21, Tikhoretsky pr., Saint-Petersburg, Russia, 194064, tel.: +7(812) 552-13-25 e-mail: zheleznyakov@rtc.ru