Alternative robot arm ML approaches

The realm of robot arm control via machine learning is really just getting started. There are a couple of research avenues I wanted to bring to your attention as you look for further study. One way to approach our understanding of robot movement is to consider the balance between exploitation and exploration. Exploitation is getting the robot to its goal as quickly as possible. Exploration is using the space around the robot to try new things. The path-planning program may have been stuck on a local minimum (think of this as a blind alley), and there could be better, more optimal solutions available that had not been considered.

There is also more than one way to teach a robot. We have been using a form of self-exploration in our training. What if we could show the robot what to do and have it learn by example? We could let the robot observe a human doing the same task, and have it try to emulate the results. Let’s discuss some alternative methods in the following sections.

Google’s SAC-X

Google is trying a slightly different approach to the robot arm problem. In their Scheduled Auxiliary Control (SAC- X) program, they surmise that it can be quite difficult to assign reward points to individual movements of the robot arm. They break down a complex task into smaller auxiliary tasks, and give reward points for those supporting tasks to let the robot build up to a complicated challenge. If we were stacking blocks with a robot arm, we might separate picking up the block as one task, moving with the block in hand as another, and so on. Google referred to this as a sparse reward problem if reinforcement was only used on the main task, stacking a block on top of another. You can imagine how, in the process of teaching a robot to stack blocks, there would be thousands of failed attempts before a successful move resulted in a reward.

Amazon Robotics Challenge

Amazon has millions and millions of boxes, parts, bits, and other things on its shelves. The company needs to get the stuff from the shelves into small boxes so they can ship it to you as fast as possible when you order it. For the last few years, Amazon has sponsored the Amazon Robotics Challenge, where teams from universities were invited to use robot arms to pick up items off a shelf and, you guessed it, put them into a box.

When you consider that Amazon sells almost everything imaginable, this is a real challenge. In 2017, a team from Queensland, Australia, won the challenge with a low-cost arm and a really good hand-tracking system.