Research Physical Human–Robot Interaction

IROS 2022pHRI · Haptic guidance & AR

Robot-Assisted Drilling on Curved Surfaces with Haptic Guidance under Adaptive Admittance Control

Alireza Madani, Pouya P. Niaz, Berk Guler, Yusuf Aydin, Cagatay Basdogan

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022

An operator wearing a HoloLens guides a KUKA iiwa with a mounted drill toward a curved wooden workpiece
The operator guides the cobot-mounted drill while the AR headset shows the drill targets.

Drilling a hole on a curved surface with a desired angle is prone to failure when done manually, due to the difficulties in drill alignment and also inherent instabilities of the task, potentially causing injury and fatigue to the workers. On the other hand, it can be impractical to fully automate such a task in real manufacturing environments because the parts arriving at an assembly line can have various complex shapes where drill point locations are not easily accessible, making automated path planning difficult. In this work, an adaptive admittance controller with 6 degrees of freedom is developed and deployed on a KUKA LBR iiwa 7 cobot such that the operator is able to manipulate a drill mounted on the robot with one hand comfortably and open holes on a curved surface with haptic guidance of the cobot and visual guidance provided through an AR interface. Real-time adaptation of the admittance damping provides more transparency when driving the robot in free space while ensuring stability during drilling. After the user brings the drill sufficiently close to the drill target and roughly aligns to the desired drilling angle, the haptic guidance module fine tunes the alignment first and then constrains the user movement to the drilling axis only, after which the operator simply pushes the drill into the workpiece with minimal effort. Two sets of experiments were conducted to investigate the potential benefits of the haptic guidance module quantitatively (Experiment I) and also the practical value of the proposed pHRI system for real manufacturing settings based on the subjective opinion of the participants (Experiment II). The results of Experiment I, conducted with 3 naive participants, show that the haptic guidance improves task completion time by 26% while decreasing human effort by 16% and muscle activation levels by 27% compared to no haptic guidance condition. The results of Experiment II, conducted with 3 experienced industrial workers, show that the proposed system is perceived to be easy to use, safe, and helpful in carrying out the drilling task.

Augmented-reality view with a target hemisphere at the drill point, and the robot-mounted drill with AR alignment bars
The AR interface: (a) target hemisphere and alignment arrow with haptic guidance, (b) alignment error bars without it.

Key results

  • 26%shorter task completion time with haptic guidance
  • 16%less human effort
  • 27%lower forearm muscle activation (sEMG)

My contributions

  • Developed an admittance controller in C++ using the Fast Robot Interface (FRI) of the KUKA iiwa cobot.
  • Developed the mixed-reality interface in C# for Microsoft HoloLens with Unity and the Mixed Reality Toolkit (MRTK).
  • Implemented a Kalman filter to track the 6-DoF motion of the HoloLens and the workpiece from AprilTag fiducial markers.
  • Built a 3D mesh model of the workpiece with a depth camera (Microsoft Kinect).
  • Set up TCP communication between the robot and the HoloLens.

Experiment

Six snapshots of the drilling protocol with haptic guidance
The drilling protocol under haptic guidance, step by step.

Video