Use of augmented reality for remote control of mobile robots in the presence of time delays in data transmission channels

Use of augmented reality for remote control of mobile robots in the presence of time delays in data transmission channels

Dmitrii S. Popov
Russian State Scientific Center for Robotics and Technical Cybernetics (RTC), Head of the Design Bureau, 21, Tikhoretsky pr., Saint Petersburg, 194064, Russia, tel.: +7(812)552-04-77, This email address is being protected from spambots. You need JavaScript enabled to view it., ORCID: 0000-0003-4575-9195

Received October 17, 2022

Abstract
The paper is devoted to the teleoperation of ground mobile robots in extreme conditions. The presence of time delays in information transmission channels has a significant impact on the efficiency of control. The total delay values can range from hundreds of milliseconds to units of seconds. Sufficiently large values are the reasons for the loss of stability of control systems. To reduce the negative impact of delays, it is proposed to use a method of supporting the operator, which consists in superimposing information about the robot's position on the image received from the video camera and displayed on the control panel at the moment when it receives the generated commands. The position calculation uses the fact that telemetry and command transmission delays are generally much less than video transmission delays. The proposed method is simple, requires a minimum amount of computing resources and can be implemented as a part of low-cost robotic systems. The presented simulation results demonstrate the effectiveness of the method.

Key words
Extreme robotics, mobile robot, teleoperation, time delay, augmented reality.

Acknowledgements
The results were obtained within the framework of the state task of Russian Ministry of Education and Science FNRG-2022-0026 1022031700007-3-2.2.2 №075-01623-22-03 «Research on ways to implement the adaptive stability of a ground-based modular reconfigurable robotic chassis of a light class when moving in a non-deterministic environment with various payloads».

DOI
10.31776/RTCJ.10405

Bibliographic description
Popov, D.N. (2022). Use of augmented reality for remote control of mobile robots in the presence of time delays in data transmission channels. Robotics and Technical Cybernetics, 10(4), pp.275-280.

UDC identifier:
681.5

References 

  1. Spassky, B.A. (2020). Teleupravlenie v ekstremal'noy robototekhnike [Teleoperation in extreme robotics]. Robototekhnika i Tekhnicheskaya Kibernetika [Robotics and Technical Cybernetics], 8(2), pp.101-111. DOI: 10.31776/RTCJ.8202. (in Russian).
  2. Ando, N., Lee, J.H. and Hashimoto, H. (1999). A study on influence of time delay in teleoperation. In: 1999 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (Cat. No. 99TH8399): proceedings of a conference, pp.317-322. DOI: 10.1109/AIM.1999.803185.
  3. Sanders, D.A. et al. (2016). Tele-operator performance and their perception of system time lags when completing mobile robot tasks. In: 2016 9th International Conference on Human System Interactions (HSI): proceedings of a conference. IEEE, pp.236-242. DOI: 10.1109/HSI.2016.7529637.
  4. Popov, D.S. (2022). Distancionnoe upravlenie nazemnymi robototekhnicheskimi kompleksami v usloviyah deystviya vremennyh zaderzhek v kanalah peredachi informacii [Teleoperation of ground-based mobile robotic systems with time delays in data transmission channels]. Robototekhnika i Tekhnicheskaya Kibernetika [Robotics and Technical Cybernetics], 10(3), pp.213-218. DOI: 10.31776/RTCJ.10306. (in Russian).
  5. Brudnak, M. (2016). Predictive displays for high latency teleoperation. In: 2016 NDIA Ground Vehicle Systems Engineering and Technology Symposium: proceedings of a symposium, pp.1-16.
  6. Chucholowski, F.E. (2016). Evaluation of display methods for teleoperation of road vehicles. Journal of Unmanned System Technology, 3(3), pp.80-85. DOI: 10.21535%2Fjust.v3i3.38.
  7. Dybvik, H. et al. (2021). A low-cost predictive display for teleoperation: Investigating effects on human performance and workload. International Journal of Human-Computer Studies, 145, 102536. DOI: 10.1016/j.ijhcs.2020.102536.
  8. Vasilieva, P.Yu. et al. (2021). Bortovoy kompleks audio-video sredstv v sostave sistem distancionnogo upravleniya nazemnymi mobil'nymi robotami [On-board complex of audio-video equipment as a part of remote control systems for ground mobile robots]. Ekstremal'naya Robototekhnika [Extreme Robotics], pp.177-196. (in Russian).
  9. Tao, Y. et al. (2021). A comparative analysis of trajectory similarity measures. GIScience & Remote Sensing, 58(5), pp.643-669. DOI: 10.1080/15481603.2021.1908927.
  10. International Federation of Robotics, (2022). President’s report by Milton Guerry, [online]. Available at: https://ifr.org/ifr-press-releases/news/presidents-report-by-milton-guerry-2-2022 (Accessed 13 July 2022).