VR-based two-arm anthropomorphic manipulation system control methods

VR-based two-arm anthropomorphic manipulation system control methods

Iuliia O. Vidimina
Russian State Scientific Center for Robotics and Technical Cybernetics, Electronics Engineer, 21, Tikhoretsky pr., Saint Petersburg, 194064, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it.

Mihail G. Fikov
Russian State Scientific Center for Robotics and Technical Cybernetics, Programmer, 21, Tikhoretsky pr., Saint Petersburg, 194064, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it.

Viktor V. Varlashin
Russian State Scientific Center for Robotics and Technical Cybernetics, Head of Laboratory, 21, Tikhoretsky pr., Saint Petersburg, 194064, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it.

Dmitrii S. Popov
Russian State Scientific Center for Robotics and Technical Cybernetics, Deputy Chief Designer, 21, Tikhoretsky pr., Saint Petersburg, 194064, Russia, This email address is being protected from spambots. You need JavaScript enabled to view it., ORCID: 0000-0003-4575-9195

Received December 19, 2023

Abstract
VR-based two-arm anthropomorphic manipulation system control methods are considered to solve tasks in small-size spaces, considering factors reducing control efficiency. Methods for developing an intuitive user interface and control methods are described considering time delays, the probability of collisions and self-intersections, and discrepancy between the actions of the operator and the manipulation system. The best ways to control a two-arm anthropomorphic manipulation system are identified. A concept for such two-arm anthropomorphic manipulation system control method based on developing an intuitive user interface with predictive visual feedback and assistive vibration feedback is proposed.

Key words
Two-armed anthropomorphic manipulation system, virtual reality technologies, feedback, digital twin.

Acknowledgements
The results were obtained within the framework of the state task of Russian Ministry of Education and Science in 2023 FNRG-2022-0012 1021051302301-9-2.2.2 №075-01595-23-04 « Development of a collaborative robot with an assisting two-armed anthropomorphic manipulation system for working in small spaces».

DOI
10.31776/RTCJ.12305

Bibliographic description
Vidimina, I.O., Fikov, M.G., Varlashin, V.V. and Popov, D.S. (2024), "VR-based two-arm anthropomorphic manipulation system control methods", Robotics and Technical Cybernetics, vol. 12, no. 3, pp. 202-211, DOI: 10.31776/RTCJ.12305. (in Russian).

UDC identifier
004.896:007.52

References

  1. Kupcov, E. A. (2012), “Semi-automatic control system for a two-armed manipulation robot”, Engineering Journal: Science and Innovation, 6 (6), p. 31. (in Russian).
  2. Sychkov, V.B. (2019), “Method for evaluating the efficiency of copying control of anthropomorphic robot manipulators”, Control, communication and security systems, 3, pp. 179-201. (in Russian).
  3. Galin, R.R. (2019), “Human-robot interaction in an ergative robotic system”, In: XIII All-Russian Meeting on Management Problems WSPU-2019, pp. 1197-1201. (in Russian).
  4. Naceri, A. et al. (2019), “Towards a virtual reality interface for remote robotic teleoperation”, In: 2019 19th International Conference on Advanced Robotics (ICAR), pp. 284–289, DOI:10.1109/icar46387.2019.8981649.
  5. Wonsick, M. and Padir, T. (2020), “A systematic review of virtual reality interfaces for controlling and interacting with robots”, Applied Sciences, 10(24), p. 9051, DOI:10.3390/app10249051.
  6. Huitaek Yun (2021), “Immersive and interactive cyber-physical system”, Abstract of the degree for doctor of philosophy School of Mechanical Engineering dissertation, Purdue University Graduate School, West Lafayette, Indiana, available at: https://hammer.purdue.edu/articles/thesis/Immersive_and_interactive_cyber-physical_system/15016110/1 (Accessed December 7, 2023).
  7. de Giorgio, A. et al. (2017), “Human-machine collaboration in virtual reality for Adaptive Production Engineering”, Procedia Manufacturing, 11, pp. 1279–1287, DOI: 10.1016/j.promfg.2017.07.255.
  8. Ostanin, M., Yagfarov, R. and Klimchik, A. (2019), “Interactive Robots Control using mixed reality”, IFAC-PapersOnLine, 52(13), pp. 695–700, DOI: 10.1016/j.ifacol.2019.11.307.
  9. Kuts, V. et al. (2019), “Digital twin based synchronised control and simulation of the industrial robotic cell using virtual reality”, Journal of Machine Engineering, 19(1), pp. 128–144, DOI:10.5604/01.3001.0013.0464.
  10. Ostanin, M. and Klimchik, A. (2018), “Interactive robot programing using mixed reality”, IFAC-PapersOnLine, 51(22), pp. 50–55, DOI: 10.1016/j.ifacol.2018.11.517.
  11. Sobhani, M. Et al. (2020), “Robot teleoperation through virtual reality interfaces: Comparing effects of fixed and moving cameras”, Vam-Hri, pp. 1-9.
  12. Chen, Y. et al. (2020), “Real-time 3D unstructured environment reconstruction utilizing VR and Kinect-based immersive teleoperation for Agricultural Field Robots”, Computers and Electronics in Agriculture, 175, 105579, DOI: 10.1016/j.compag.2020.105579.
  13. Pace, F.D. et al. (2021), “Leveraging enhanced virtual reality methods and environments for efficient, intuitive, and immersive teleoperation of Robots”, In: 2021 IEEE International Conference on Robotics and Automation (ICRA), 12967-12973, DOI: 10.1109/icra48506.2021.9560757.
  14. Ganyushkin, A.L. and Zuev, I.A., Limited Liability Company “ROBOT” (2020), Sposob upravlenija robotom s pomoshh'ju kontrollerov i shlema virtual'noj real'nosti [Method of robot control using controllers and virtual reality helmet], Russia. (in Russian).
  15. Kulakov, F.M. (2016), “Telecontrol of space robots’ Izvestia of the Russian Academy of Sciences”, Theory and control systems, 4, pp. 141-192. (in Russian).
  16. Cash, H. and Prescott, T.J. (2019), “Improving the visual comfort of virtual reality telepresence for robotics”, Social Robotics, pp. 697-706, DOI: 10.1007/978-3-030-35888-4_65.
  17. Kugurakova, B.V. et al (2022), “Remote control of a robotic device using virtual reality technologies”, Software Products and Systems, 35(3), pp. 348-361. (in Russian).
  18. Kheddar, A. et al. (2007), “Enhanced teleoperation through virtual reality techniques”, Springer Tracts in Advanced Robotics, pp. 139-159, DOI: 10.1007/978-3-540-71364-7_10.
  19. Starostin, A.G., Afanasyev, V.V. and Ignatenko, A.V. (2016), “Remote presence system using virtual reality helmet”, In: GRAPHICON'2016, pp. 383-387. (in Russian).
  20. Dudorov, E. A. (2021), “Development of algorithms for robot motion control taking into account the time”, In: Perspektivnye sistemy i zadachi upravlenija [Perspective Systems and Control Problems], pp. 265-270. (in Russian).
  21. Mostefa, M. et al. (2015), “Design of Mobile Robot teleoperation system based on virtual reality”, In: 2015 3rd International Conference on Control, Engineering & Information Technology (CEIT), pp. 1-6. DOI: 10.1109/ceit.2015.7233180.
  22. Leeper, A. et al. (2013), “Methods for collision-free arm teleoperation in clutter using constraints from 3D sensor data”, In: 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), pp. 520-257. DOI: 10.1109/humanoids.2013.7030023.
  23. Coffey, M. and Pierson, A. (2022), “Collaborative teleoperation with haptic feedback for collision-free navigation of Ground Robots”, In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8141-8148, DOI: 10.1109/iros47612.2022.9981426.
  24. Paiusco, A. (2023), “Bimanual robotic manipulation based on potential fields”, Master Thesis in Computer Engineering, University of Padova, Italy.
  25. Szczurek, K.A. et al. (2022), “Mixed reality human–robot interface with adaptive communications congestion control for the teleoperation of mobile redundant manipulators in Hazardous Environments”, IEEE Access, 10, pp. 87182–87216, DOI: 10.1109/access.2022.3198984.
  26. Zhang, D. and Khurshid, R.P. (2019), “Variable-scaling rate control for collision-free teleoperation of an unmanned aerial vehicle”, arXiv:1911.04466, DOI: 10.48550/arXiv.1911.04466.
  27. Vorobyov, E.I. and Khatuntsev, D.I. (2016), “Two-armed robots. Peculiarities of motion control algorithms construction’ Automation”, Modern technologies, 3, pp. 19-23. (in Russian).
  28. de Barros, P.G. and Lindeman, R.W. (2013), “Performance effects of multi-sensory displays in virtual teleoperation environments”, In: Proceedings of the 1st symposium on Spatial user interaction, pp. 41-48. DOI: 10.1145/2491367.2491371.
  29. Yashin, G.A. et al. (2019), “AeroVr: Virtual reality-based teleoperation with tactile feedback for aerial manipulation”, In: 2019 19th International Conference on Advanced Robotics (ICAR), pp. 767-772, DOI: 10.1109/icar46387.2019.8981574.
  30. Pacchierotti, C. et al. (2015), “Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems”, The International Journal of Robotics Research, 34(14), pp. 1773-1787, DOI:10.1177/0278364915603135.
  31. Omarali, B. et al. (2021), “Workspace scaling and rate mode control for virtual reality based robot teleoperation”, In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 607–612, DOI: 10.1109/smc52423.2021.9658796.
  32. Florent, A.P. et al (2023), “TELESIM: A Modular and Plug-and-Play Framework for Robotic Arm Teleoperation using a Digital Twin”, arXiv - CS – Robotics, DOI: arxiv-2309.10579.
  33. Fikov, M.G. et al. (2022), “Study of the characteristics of the manipulator collision avoidance system”, Robotics and technical cybernetics, 10(4), pp. 287-292. (in Russian).