Collaborative manipulation robots programming with the use of augmented reality interface

Collaborative manipulation robots programming with the use of augmented reality interface

Alexander Schwandt
FKS-Maschinenbau GmbH, Development Division, Engineer, Head of Development Division, 83-85, Wilhelminenhof street, Berlin, D-12459, Germany, tel.: +49(30)551507900, This email address is being protected from spambots. You need JavaScript enabled to view it.

Arkadij S. Yuschenko
Doctor of Technical Science, Bauman Moscow State Technical University (BMSTU), Professor, 5-1, 2-ya Baumanskaya ul., Moscow, 105005, Russia, tel.: +7(499)263-60-54, This email address is being protected from spambots. You need JavaScript enabled to view it.


Received 12 February 2020

Abstract
The integration of robotics in the process of production automation not only at large, but also at small-scale, lightly convertible, enterprises, has led to an increasingly widespread expansion of collaborative robotics (CoRT). In this case, the complete safety of the industrial robot in relation to a human located in the same workspace is provided, but the exploitation operations of the industrial robot, especially its programming, are simplified to the maximum. In fact, it doesn’t require from operator any additional knowledge or skills related to robot control. In most cases, the operator can simply move the industrial manipulator (IM) grip along a predetermined path and this movement will be remembered, i.e. IM programming is carried out by demonstration. Nevertheless, this method is still associated with a certain risk, when it comes to a sufficiently large IM. In addition, this method is not applicable when performing operations that require sufficient accuracy. Difficulties appear also when performing assembly operations. The application of the augmented reality interface for programming collaborative manipulation robots is proposed in this paper. The proposed method can be named virtual IM programming by demonstration. In this case motion programming occurs without turning on the IM and with use of the 3D model of the manipulator’s grip or an object held in the grip, the working tool etc. This model’s motion is performed by the operator, the last observes through miniature screen of transparent glasses. At the same time, the operator sees the surrounding reality, which is necessary for the correct choice of movement and allows avoiding possible collisions. In difficult situations, the operator uses their own intuition, experience when programming the movement, which allows us to call this method “intuitive programming”. After the operation is completed quite satisfactorily from the operator’s point of view and according to objective indicators, that can also be measured, the IM turns on and repeats the given operation or sequence of operations when the human operator has already left the operational field. A technical solution to the problem is proposed, as well as preliminary studies on the accuracy of this CoRT programming method.

Key words
Manipulation robot, human-robot interface, augmented reality, programming by demonstration, intuitive programming, collaborative robotics.

Acknowledgements
This work was financially supported by the Russian Foundation for Basic Research (project no.18-07-0131). The authors also express gratitude to the management of FKS-Maschinenbau GmbH (Federal Republic of Germany, Berlin) for their support in conducting the research.

DOI
https://doi.org/10.31776/RTCJ.8205

Bibliographic description
Schwandt, A. and Yuschenko, A., 2020. Collaborative manipulation robots programming with the use of augmented reality interface. Robotics and Technical Cybernetics, 8(2), pp.139-149.

UDC identifier:
004.896

References

  1. Akan, B., Ameri, A., Curuklu, B. and Asplund, L. (2011). Intuitive Industrial Robot Programming Through Incremental Multimodal Language and Augmented Reality. In: Proceedings of the IEEE International Conference on Robotics and Automation. Pp. 3934-3939.
  2. Petterson, T., Pretlove, J., Scorub, C., Engedal, T. and Lokstad, T. (2006). Augemted reality for programming industrial robots. In: Proceedings of The Second IEEE and ACM International Symposium on Mixed and Augmented Reality. Pp. 319-320.
  3. Po, C.-W., Wang, R., Kin, K., Twigg, C., Han., S., Yang, M.-H. and Chien, S.-Y. (2017). Accurate 6Dof Tracking of a Passive Stylus. In: Proceedings of The 30th Annual ACM Symposium on User Interface Software and Technology. Pp.365-374.
  4. Wandelbots.com. n.d. Description of Natural Programming Interface «Tracepen». [online] Available at: <https://www.wandelbots.com/tracepen/> [Accessed 1 February 2020].
  5. Olwall, A., Gustafsson, J. and Lindfors, C. (2008). Spacial augmented reality on industrial CNC-machines. In: Proceedings of SPIE. Pp. 70-78.
  6. Chong J.W.S., Ong S.K., Nee A.Y.-C. and Youcef-Youmi K. (2009). Robot programming using augmented reality: An interactive method for planning collision-free paths. In: Proceedings of IEEE.
  7. Marin, R., Sanz, P. and Sanchez, J. (2002). A very high level interface to teleoperate a robot via Web including augmented reality. In: Proceedings of the IEEE International Conference on Robotics and Automation. Pp. 2725-2730.
  8. Jara, C.A., Candelas, F.A., Gil, P., Fernandez, M. and Torres, F. (2005). An augmented reality interface for training robotics through the web. In: Conference Proceedings. Pp. 189-194.
  9. Zenkievich, S.L. and Yuschenko, A.S. (2004). Osnovy Upravleniya Manipulyatsionnymi Robotami [The Basic of Manipulation Robots Control]. BMSTU, pp. 51-57. (in Russian).
  10. Siciliano, B. and Khatib, O. (2008). Handbook of Robotics. Berlin-Heidelberg: Springer. Pp. 323-328.
  11. Docs.opencv.org. 2020. Camera Calibration and 3D Reconstruction. [online] Available at: <https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html> [Accessed 1 February 2020].
  12. Yuschenko, A.S. (2018). Human and robot – the ergonomic problems of collaborative robotics. World of psychology, 4(96), pp. 86-102.
  13. Mooring, B., Yvi, R. and Morris D. (1991). Circle Point Analysis - The Measurement Phase. In: Fundamental of manipulator Calibration. John Wiley & Sons Inc. Pp. 175-182.
  14. Haun, M. (2007). Arten der Robotersystemprogrammierung. In: Handbuch Robotik.  Berlin-Heidelberg: Springer. Pp. 173-180.
Editorial office address: 21, Tikhoretsky pr., Saint-Petersburg, Russia, 194064, tel.: +7(812) 552-13-25 e-mail: zheleznyakov@rtc.ru