Final Report Summary - SKILLASSIST (Skill Assistance with Robot for Manual Welding)
1) PUBLISHABLE SUMMARY
Project Objectives:
The goal of this project was to develop a quantifiable measure of skills required to perform complex tasks that are typical of manual welding in the industry and use this knowledge for robotic assistance and training purposes. Airbrush painting was also used as a test platform to quantify the skill difference between dominant and non-dominant hand performances, prior to the welding experiments. Automation of welding is not always possible due to the complexity and variety of welding tasks. Many industries still rely on manual welding. This project aimed to link the human assistive robotics research with the industrial manufacturing, particularly with manual welding. The project aimed, first, to develop an understanding of the differences between professional and novice welders in terms of hand-impedance and position variation measurements, and, then, to develop two robotic systems, one for assisting manual welding and one for training of novice welders with a robot. The basic objectives of the project were as follows:
1) Performing impedance measurements across professional and novice welders while doing welding interactively with a robot, as well as across dominant versus non-dominant hands while doing airbrush painting with the same system.
2) Identifying the differences between professional and novice welders, as well as dominant versus non-dominant hand performances, in terms of impedance and position variations.
3) Developing a robotic assistance scheme for manual welding as well as for airbrush painting.
4) Developing a robotic training system for manual welding.
Work Performed:
The following items of work were performed within the project period:
1) An interactive and shared controlled robotic welding/painting setup was developed using the KUKA LWR 4+ robot, where the system could introduce force disturbances for impedance measurement purposes during welding/painting.
2) Impedance measurements were performed and directional damping type robotic assistance was implemented with subjects while they were doing airbrush painting with the interactive system with their dominant and non-dominant hands (Fig. 1a). Dominant and non-dominant hand paintings corresponded respectively to skilled and unskilled manipulations and served as a preliminary test case prior to the welding experiments with professional and novice welders.
3) Impedance measurements were performed with professional and novice welders when they were doing TIG welding interactively with the robot (Fig. 1b).
(a) (b)
Fig. 1: (a) Airbrush painting interactively with KUKA LWR. (b) TIG welding interactively with KUKA LWR.
4) An impedance compensation type robotic assistance was developed for manual welding by using the knowledge of hand-impedance characteristics of professional and novice welders. The assistive scheme (Fig. 2) estimates the intended welding direction in real-time using a smooth Kalman filter and compensates the inferior level of hand-impedance of the novice welders in the perpendicular directions. The scheme was applied and tested by professional and novice welders. The assistive scheme was applied also for airbrush painting with dominant and non-dominant hands.
Fig. 2: Impedance compensation type robotic assistance integrated with the admittance control of the robot. [Parameters: fh: human force; mt: mass of the torch; ft: inertial force of the torch; fs: force sensor reading; fc: commanded force; mv: virtual mass at the end effector; pd: desired position command; s: Laplace derivator; vd: desired velocity command; pr: actual robot position; M: compensated mass; D: compensated damping; K: compensated stiffness; fv: compensation force; fa: actual assistive force.]
5) A training system was developed for manual welding by integrating a LED and a buzzer to the welding helmet (Fig. 3). The training setup estimates the intended welding direction, detects the deviations from the estimated direction due to hand tremor, and generates visual or audio alarms as feedback to notice the welder. The training system was applied and tested by professional and novice welders.
Fig. 3: The helmet is equipped with a LED and a buzzer in order to give alarms in the form of a flashing light and beep sound for training.
Main Results Achieved:
The below four items of results were achieved with the above described experiments. The number of subjects who participated in each experiment is indicated in the tables and the figure caption given below. In the airbrush experiments the subjects performed straight line painting with their dominant and non-dominant hands. In the welding experiments the professional and novice welders performed Tungsten Inert Gas (TIG) welding on the connecting edges of two stainless steel plates. In all cases the subjects were left free to orient their arms and body as they found convenient. They were instructed to aim at the best performance they could.
1) While doing airbrush painting, subjects applied larger damping with their dominant hands in the direction perpendicular to the painting line (Table 1). Robotic assistance by compensating damping in the directions perpendicular to the painting line improved painting quality (Fig. 4).
Table 1: Average hand-impedance parameters while subjects performed airbrush painting in y direction.
(a) (b)
Fig. 4: Sample painting with non-dominant hand (a) without and (b) with robotic assistance.
2) Professional welders applied larger impedance (rate-hardness) compared to novice welders. The most significant difference occurred in the damping parameter in the direction perpendicular to the welding line on the metal plate (Table 2).
Table 2: Average hand-impedance measures while subjects performed TIG welding with the robot.
3) Impedance compensation type robotic assistance improved welding quality of the novice welders by significantly decreasing the position variation of the torch. Answers to user questionnaire showed that all novice welders and most of the professional welders found welding with robotic assistance easier and more successful than welding without the robot.
Table 3: Average ratings of the responses of novice and professional welders to the user questionnaire.
4) With the training system, the position variations of the novice welders decreased when they received immediate notice feedback for the hand vibrations. The most significant decrease occurred when the notice feedback was in the form of a visual alarm (flashing LED) compared to an audio alarm (beep sound) (Fig. 5).
Fig. 5: Position variations of the tip of the welding torch in the presence of no alarms, sound alarms, and light alarms. Asterisk indicates statistically significant difference. 12 novice and 5 professional welders participated.
Potential Impact:
The impact of the results of this project will be substantial for assistive robotics. This project provided an answer to the questions of “what, when, and how to assist with robot” during a fine and industrially relevant manipulation task, manual welding: the impedance level is to be compensated with virtual dynamics, when the performer generates large variation movements. Furthermore, the project also demonstrated that providing real-time feedback alarm results in immediate improvement of welding performance by decreasing the position variations, and potentially helps the novice welders to learn suppression of such variations in the long run. The developed assistance and training schemes and the method based on impedance measurements may inspire the design and development of assistive control systems for a variety of manipulation tasks ranging from painting, polishing, scrubbing in industry, to micro and minimally invasive surgery and physiological rehabilitation in medicine. The method also exemplified a bio-inspired methodology, by which one first analyzes the human behavior, identifies human hand impedance and position variations to determine skill levels in manipulation, and finally uses these to determine control characteristics for robotic assistance and training purposes.
Project Objectives:
The goal of this project was to develop a quantifiable measure of skills required to perform complex tasks that are typical of manual welding in the industry and use this knowledge for robotic assistance and training purposes. Airbrush painting was also used as a test platform to quantify the skill difference between dominant and non-dominant hand performances, prior to the welding experiments. Automation of welding is not always possible due to the complexity and variety of welding tasks. Many industries still rely on manual welding. This project aimed to link the human assistive robotics research with the industrial manufacturing, particularly with manual welding. The project aimed, first, to develop an understanding of the differences between professional and novice welders in terms of hand-impedance and position variation measurements, and, then, to develop two robotic systems, one for assisting manual welding and one for training of novice welders with a robot. The basic objectives of the project were as follows:
1) Performing impedance measurements across professional and novice welders while doing welding interactively with a robot, as well as across dominant versus non-dominant hands while doing airbrush painting with the same system.
2) Identifying the differences between professional and novice welders, as well as dominant versus non-dominant hand performances, in terms of impedance and position variations.
3) Developing a robotic assistance scheme for manual welding as well as for airbrush painting.
4) Developing a robotic training system for manual welding.
Work Performed:
The following items of work were performed within the project period:
1) An interactive and shared controlled robotic welding/painting setup was developed using the KUKA LWR 4+ robot, where the system could introduce force disturbances for impedance measurement purposes during welding/painting.
2) Impedance measurements were performed and directional damping type robotic assistance was implemented with subjects while they were doing airbrush painting with the interactive system with their dominant and non-dominant hands (Fig. 1a). Dominant and non-dominant hand paintings corresponded respectively to skilled and unskilled manipulations and served as a preliminary test case prior to the welding experiments with professional and novice welders.
3) Impedance measurements were performed with professional and novice welders when they were doing TIG welding interactively with the robot (Fig. 1b).
(a) (b)
Fig. 1: (a) Airbrush painting interactively with KUKA LWR. (b) TIG welding interactively with KUKA LWR.
4) An impedance compensation type robotic assistance was developed for manual welding by using the knowledge of hand-impedance characteristics of professional and novice welders. The assistive scheme (Fig. 2) estimates the intended welding direction in real-time using a smooth Kalman filter and compensates the inferior level of hand-impedance of the novice welders in the perpendicular directions. The scheme was applied and tested by professional and novice welders. The assistive scheme was applied also for airbrush painting with dominant and non-dominant hands.
Fig. 2: Impedance compensation type robotic assistance integrated with the admittance control of the robot. [Parameters: fh: human force; mt: mass of the torch; ft: inertial force of the torch; fs: force sensor reading; fc: commanded force; mv: virtual mass at the end effector; pd: desired position command; s: Laplace derivator; vd: desired velocity command; pr: actual robot position; M: compensated mass; D: compensated damping; K: compensated stiffness; fv: compensation force; fa: actual assistive force.]
5) A training system was developed for manual welding by integrating a LED and a buzzer to the welding helmet (Fig. 3). The training setup estimates the intended welding direction, detects the deviations from the estimated direction due to hand tremor, and generates visual or audio alarms as feedback to notice the welder. The training system was applied and tested by professional and novice welders.
Fig. 3: The helmet is equipped with a LED and a buzzer in order to give alarms in the form of a flashing light and beep sound for training.
Main Results Achieved:
The below four items of results were achieved with the above described experiments. The number of subjects who participated in each experiment is indicated in the tables and the figure caption given below. In the airbrush experiments the subjects performed straight line painting with their dominant and non-dominant hands. In the welding experiments the professional and novice welders performed Tungsten Inert Gas (TIG) welding on the connecting edges of two stainless steel plates. In all cases the subjects were left free to orient their arms and body as they found convenient. They were instructed to aim at the best performance they could.
1) While doing airbrush painting, subjects applied larger damping with their dominant hands in the direction perpendicular to the painting line (Table 1). Robotic assistance by compensating damping in the directions perpendicular to the painting line improved painting quality (Fig. 4).
Table 1: Average hand-impedance parameters while subjects performed airbrush painting in y direction.
(a) (b)
Fig. 4: Sample painting with non-dominant hand (a) without and (b) with robotic assistance.
2) Professional welders applied larger impedance (rate-hardness) compared to novice welders. The most significant difference occurred in the damping parameter in the direction perpendicular to the welding line on the metal plate (Table 2).
Table 2: Average hand-impedance measures while subjects performed TIG welding with the robot.
3) Impedance compensation type robotic assistance improved welding quality of the novice welders by significantly decreasing the position variation of the torch. Answers to user questionnaire showed that all novice welders and most of the professional welders found welding with robotic assistance easier and more successful than welding without the robot.
Table 3: Average ratings of the responses of novice and professional welders to the user questionnaire.
4) With the training system, the position variations of the novice welders decreased when they received immediate notice feedback for the hand vibrations. The most significant decrease occurred when the notice feedback was in the form of a visual alarm (flashing LED) compared to an audio alarm (beep sound) (Fig. 5).
Fig. 5: Position variations of the tip of the welding torch in the presence of no alarms, sound alarms, and light alarms. Asterisk indicates statistically significant difference. 12 novice and 5 professional welders participated.
Potential Impact:
The impact of the results of this project will be substantial for assistive robotics. This project provided an answer to the questions of “what, when, and how to assist with robot” during a fine and industrially relevant manipulation task, manual welding: the impedance level is to be compensated with virtual dynamics, when the performer generates large variation movements. Furthermore, the project also demonstrated that providing real-time feedback alarm results in immediate improvement of welding performance by decreasing the position variations, and potentially helps the novice welders to learn suppression of such variations in the long run. The developed assistance and training schemes and the method based on impedance measurements may inspire the design and development of assistive control systems for a variety of manipulation tasks ranging from painting, polishing, scrubbing in industry, to micro and minimally invasive surgery and physiological rehabilitation in medicine. The method also exemplified a bio-inspired methodology, by which one first analyzes the human behavior, identifies human hand impedance and position variations to determine skill levels in manipulation, and finally uses these to determine control characteristics for robotic assistance and training purposes.