Development of technical appearance of human-machine interface for group control of unmanned robots when performing agricultural tasks

Development of technical appearance of human-machine interface for group control of unmanned robots when performing agricultural tasks

Anna I. Motienko
PhD in Technical Sciences, Saint Petersburg Federal Research Center of the Russian Academy of Sciences, laboratory of Big Data Technologies of Sociocyberphysical Systems, Senior Research Scientist, 39, 14 line V.O., Saint Petersburg, 199178, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it., ORCID: 0000-0002-0315-9485

Irina V. Vatamaniuk
Saint Petersburg Federal Research Center of the Russian Academy of Sciences, Laboratory of Autonomous Robotic Systems, Junior Research Scientist, 39, 14 line V.O., Saint Petersburg, 199178, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it., ORCID: 0000-0001-5388-8152

Anton I. Saveliev
PhD in Technical Sciences, Saint Petersburg Federal Research Center of the Russian Academy of Sciences, Laboratory of Autonomous Robotic Systems, Head of Laboratory, 39, 14 line V.O., Saint Petersburg, 199178, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it., ORCID: 0000-0003-1851-2699

Received 08 September 2021

Abstract
One of the promising areas of using robotic systems (RS) is group control of robots. The means of ensuring the operator's interaction with autonomous RS for monitoring and remote control - the human-machine interface (HMI) of the robotic systems, are discussed in the paper. The foreign and domestic publications describing existing approaches and priority directions to the construction of HMIs for various types of robotic systems - ground and air, as well as modern commercial solutions in this area are analyzed in the paper. The performed analysis allows us to conclude that the most popular and convenient type of human-machine interface for the types of robotic systems under consideration today is a digital application with a graphical interface. To implement complex scenarios of human-machine interaction, combinations of several methods are used, such as: a digital application with a graphical interface, virtual reality tools, gesture recognition, voice commands. The technical appearance of the man-machine interface of group unmanned robots based on the results of the analysis was developed. It ensures the prompt formulation of the applied problem to be solved and the subsequent control of its execution by the operator. The proposed solution is intuitive and flexible, which provides the user with a higher speed of information processing.

Key words
Group robotics, human-machine interface (HMI), group control of robots, HMI architecture, robotic system.

DOI
10.31776/RTCJ.9407

Bibliographic description
Motienko, A, Vatamaniuk, I. and Saveliev, A., 2021. Development of technical appearance of human-machine interface for group control of unmanned robots when performing agricultural tasks. Robotics and Technical Cybernetics, 9(4), pp.299-311.

UDC identifier:
004.5

References  

  1. gov.ru, 2020. Perspektivnye napravleniya primeneniya robototekhniki v biznese [Promising areas of application of robotics in business]. Available at: <https://digital.gov.ru/uploaded/presentations/20200325idoklad.pdf> (Accessed 12 November 2021).
  2. Gorodetsky, V.I. and Bukhvalov, O.L., 2017. Model and architecture of infrastructure for collective robotic control system. Robotics and Technical Cybernetics, 1, pp.33-44. (in Russian).
  3. Ju, C. and Son, H. I., 2019. A distributed swarm control for an agricultural multiple unmanned aerial vehicle system. Journal of Systems and Control Engineering, 233(10), pp.1298-1308.
  4. Uryasheva, A. et al., 2019. DroneGraffiti: Autonomous Multi-UAV Spray Painting. In: ACM SIGGRAPH 2019, pp.1-2.
  5. Juan, J.R. et al., 2018. Analyzing and improving multi-robot missions by using process mining. Autonomous Robots, 42(6), pp.1187–1205.
  6. Roldán, J.J. et al., 2017. Multi-robot interfaces and operator situational awareness: Study of the impact of immersion and prediction. Sensors, 17(8), p.1720.
  7. Itkin, M., Kim, M. and Park, Y., 2016. Development of cloud-based UAV monitoring and management system. Sensors, 16(11), p.1913.
  8. com, u.d. Auterion Enterprise PX4. Available at: <https://auterion.com/drone-manufacturers/enterprise-px4> (Accessed 22 November 2021).
  9. com, u.d. Visionair - GCS Software. Available at: <https://www.uavnavigation.com/
    products/ground-control-station/visionair-gcs-software> (Accessed: 22 November 2021).
  10. Sabatini, R., Lim, Y. and Gardi, A., 2018. Human-Machine Interfaces and Interactions for Multi UAS Operations. In: Proceedings of the 31th Congress of the International Council of the Aeronautical Sciences (ICAS 2018), pp.9–14.
  11. Friedrich, M. and Lieb, J., 2019. A Novel Human Machine Interface to Support Supervision and Guidance of Multiple Highly Automated Unmanned Aircraft. In: Proceedings of IEEE/AIAA 38th Digital Avionics Systems Conference (DASC), p.1–7.
  12. Ilbeygi, M. and Kangavari, M.R., 2019. A New Single-Display Intelligent Adaptive Interface for Controlling a Group of UAVs. Journal of AI and Data Mining, 7(2), pp.341–353.
  13. aero, u.d. PO dlya upravleniya BVS [Software for control of UAV]. Available at: <https://supercam.aero/catalog/po-dlya-upravleniya-bvs> (Accessed 22 November 2021).
  14. Mueller, J.B. et al., 2017. A Human-System Interface with Contingency Planning for Collaborative Operations of Unmanned Aerial Vehicles. In: Proceedings of AIAA 2017-1296. Session: Human Automation Interaction. DOI: 10.2514/6.2017-1296.
  15. Roldán, J.J. et al., 2019. Bringing Adaptive and Immersive Interfaces to Real-World Multi-Robot Scenarios: Application to Surveillance and Intervention in Infrastructures. IEEE Access, 7, pp.86319-86335.
  16. Gromov, B., Gambardella, L.M. and Di Caro, G.A., 2016. Wearable multi-modal interface for human multi-robot interaction. In: Proceedings of 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp.240–245.
  17. Patel, J., Xu, Y. and Pinciroli, C., 2019. Mixed-granularity human-swarm interaction. In: Proceedings of 2019 International Conference on Robotics and Automation (ICRA), pp.1059–1065.
  18. Mobile-industrial-robots.com, u.d. MiR Robot Interface Reference guide. Available at: <https://www.mobile-industrial-robots.com/media/2806/mir-robot-interface-20-reference-guide-v13-en.pdf> (Accessed 22 November 2021).
  19. Sergeev, S.F., 2020. Proektirovanie Interfeysov [Interface Design]. ООО «Izdatelstvo VVM» Publ., p.132. (in Russian).
  20. Chaikalis, D., Khorrami, F. and Tzes, A., 2020. Adaptive Control Approaches for an Unmanned Aerial Manipulation System. In: Proceedings of 2020 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 498–503.
  21. Tzafestas, S.G., 2010. Supervisory and Distributed Control in Automation. In: Human and Nature Minding Automation, pp.83-108.
  22. Zhang, P., 2010. Human-machine interfaces. In: Advanced Machine Interface, pp.527-55.
  23. Motlagh, F.Y. and Göhner, P., 2014. Adaptive Human-Machine-Interface of Automation Systems. In: Proceedings of Doctoral Conference on Computing, Electrical and Industrial Systems, pp.175–182.
  24. Shugaev, A., 2015. Aktual'nye tendentsii v razvitii sredstv cheloveko-mashinnogo interfeysa v proizvodstvennoy sfere [Current trends in the development of human-machine interface tools in the manufacturing sector]. Available at: <https://controlengrussia.com/cheloveko-mashinny-j-interfejs-hmi/magelis-gtu/> (Accessed 15 November 2021).
Editorial office address: 21, Tikhoretsky pr., Saint-Petersburg, Russia, 194064, tel.: +7(812) 552-13-25 e-mail: zheleznyakov@rtc.ru