University of Washington engineers and computer scientists give assembly line robots a helping hand

Robots are an indispensable part of the modern assembly line, whether it’s to mount a steering wheel or pick up packages for shipping at a distribution center. But what if you wanted to shift gears to manufacture a new product? Now that presents a challenge since robotic, hand-like grippers aren’t nimble enough to pick up and move objects of varying shapes and sizes on an assembly line.

A team of engineers and computer scientists at the University of Washington tackled this problem by using computer software to generate instructions which could be fed into a 3D printer to make grippers, each optimally designed to grab and rotate nearly two dozen different shapes, such as a tennis ball, mustard bottle and a toy-sized chair. Jeffrey Lipton is an assistant professor of mechanical engineering at the University of Washington. He joins us to talk about this research which was presented at the annual Association for Computer Machinery conference earlier this month.

Note: The following transcript was created by a computer and edited by a volunteer.

Dave Miller: This is Think Out Loud on OPB, I’m Dave Miller. Robots are very good at doing the same thing over and over and over again. It is the very heart of automation. But what if you need to pivot and change their jobs, have them, say, pick up and move a differently shaped object? How can you do that efficiently and cheaply? That is the problem that a group of engineers and computer scientists at the University of Washington set out to solve. Jeffrey Lipton is a member of that team. He is an assistant professor of mechanical engineering at U Dub, and he joins us now. Jeffrey Lipton, welcome.

Jeffrey Lipton: Thank you.

Miller: So the project came out of a problem that you saw with manufacturing during the early days of the pandemic. What happened?

Lipton: So in the early days of the pandemic, a team of colleagues around UW got together to assist with the VA and other groups in developing PPE that could be 3D printed and delivered to need. So you could kind of imagine, if you remember back then, all of a sudden we needed masks, we needed face shields, we needed different devices, and we didn’t have the ability to bring online manufacturing capacity quickly enough. And the analogy you could think to is the power grid. If you have electricity, you have a baseline load. Demand can peak and you can bring on more plants. We only had 3D printing as this peaking ability in manufacturing, and we realized we really needed some ability to complement that with assembly, where you could assemble items together, pivot an assembly line to change its need on demand when you get a sudden surge in demand, like around the early stages of the COVID-19 pandemic.

Miller: The pandemic was obviously, thankfully, a once in a century human reality. How common is it for a factory to need to quickly change their processes, meaning robots now having to quickly manipulate new objects?

Lipton: Right now, we don’t use robotics for that. But there’s actually a huge amount of study into what we call the long tail of manufacturing. There are a large number of products that are produced in small batches, one offs, unique runs that have a lot of value behind them. But you can’t automate their production because there’s no ability to take your manufacturing line and pivot it to do something else. So today, robots do the same task over and over again and work well in, say, auto plants. But if you want to change from going from producing cars like Ford did to producing face shields, you can’t take your capital equipment and just turn it on a dime.

Miller: You focused on the gripping piece of this, a robot holding onto an object. Why is that such a physical challenge?

Lipton: The things that humans find easy, computers often find quite difficult. And it’s often vice-versa, a computer can calculate stupendously large numbers, but figuring out how to hold a banana is actually very tricky for a computer to do. And so we looked at gripping as that first step, because it’s really an enabling component of all of your automation inside of a factory. You have to be able to pick up an item in order to be able to manipulate it, to scan it, to do other things with it. So once we figured out how to pick items up, we can start incorporating in more manufacturing techniques around it.

Miller: What is it about gripping, whether it’s a banana or a coffee cup or a truck chassis, why is it challenging?

Lipton: It’s a 3D spatial reasoning problem that really brings together information about the gripper and information about the object and what you intend to do around it. And that makes it really hard, because there’s just so many different ways of bringing these two objects together, and so many of them are terrible. But we can look at a coffee cup and know I should grab it by the handle, or grab it by the base. But a computer doesn’t have that ability to do spatial reasoning, and so it really just has to look at this huge number of possible ways to bring them together, and most of them lead to the cup falling over.

Miller: You’re working on something called a passive gripper. What does that mean?

Lipton: We wanted to look at how you could pick up an object without any additional movement besides just moving the arm. So we didn’t have any actuators in there to cause squeezing or any suction cups to pull things together. We just looked at the shape of the gripper. And you may say that’s very limiting, but if you think about it, most of the times you’re holding an object, you’re not squeezing it very hard or even at all. Most of the time you’re just shaping your hand to fit the object. And so if we can figure out the exact shape to pick up an object, we can do it without any additional components we need to install. And in, fact we could retool an entire robot by just giving it access to a $200 3D printer to print off components.

Miller: When I was thinking about your work this morning, I had an empty coffee cup in front of me. And having read about the work you did, I took my three fingers and put them in various configurations to try to safely, without dropping a coffee cup, just use three static fingers to do it. And I couldn’t do it, it was really hard. I’m confused how you can get a robot to pick up something even as simple as a coffee cup with just three appendages that don’t move?

Lipton: Well if you think about it, most chairs that you sit on have four legs, you can find stools that have three. And three is really that minimum number. Often, we like to use a lot more contacts when we’re picking something up. We’ve got a lot of soft pads on our hand, we can shape our hand to it, and we can maybe pick up with it touching over the entire area. But if you’re very careful in picking the right three points, it’ll balance just right. And so once we can look at all the configurations of points, we can find the sets that will support it, and then build the gripper to support those three points.

Miller: So how did you actually teach the computer how to do that? How do you do the imaging of this 3D object, which then lets you figure out how to design the gripper?

Lipton: So we always start with the CAD models. We’ve got some CAD model that we got either because we designed it in CAD and 3D printed it, or there are databases available such as the YCB database. Once we start with the CAD in the computer, we then do a bunch of simulations and say which three points can support it. And we find thousands and thousands of combinations, and whittle them down to just the ones that will support it, and more importantly, the ones that will pull away. Because you can imagine holding a ball if I supported on three points, or even six points, if I cage it in, it can’t move, but it’s perfectly supported. If I support with just three on the bottom, I can always pull away and grab it. Once we find that movement that lets us pull away from an object, we then figure out what’s the skeleton of fingers that connect those three points to the wrist, and how will they move so that they don’t hit the object.

Once we’ve got all those components together, we feed it into a program called Topology Optimization, and it gives us a single part that we can 3D print to grab that object.

Miller: What actual objects did you use for this research?

Lipton: We looked at a large number of objects. 22 were in our study. They really broke down to a couple of groups. One were common household items in this database we call YCB. It includes a mustard bottle, cups, bowls, oranges, drills. Another one was shapes we designed that were meant to be tricky or show off some difficulty in finding an algorithm. And the other ones were objects from other people’s research that were specifically designed to be hard to pick up. And we wanted to see if we could pick up the ones that other people would have challenges with.

Miller: What in general are the things that are trickiest to pick up?

Lipton: Most grippers that are used in industry today come in two forms, basically a suction cup, and a pinching motion. You can imagine a pair of tongs that you use on a grill or for salad. And so with most of those things, you need either a big flat surface, or something with two sides that are fairly smooth next to each other. And so objects that are really difficult to pick up have kind of funky knotted geometries, different holes in places, and make it very difficult to either squeeze or get a suction cup onto. And so those objects that are designed to be hard to squeeze to pick up that only use two points, we could find shapes that would lock into them and pick them up with ease.

Miller: When you say lock in, how secure does this grip have to be? Again, we’re not talking about squeezing something firmly. We’re talking about having a kind of rest in a non moving three fingered cradle. Can you tip it upside down? Can you shake it? Can you move it quickly?

Lipton: It really depends on the object. Some of the ones we found were very secure. There’s the shape known as the Stanford bunny that we did a grip on -

Miller: [laughing] Sorry, there’s a famous shape called the Stanford bunny?

Lipton: Yes, there’s a famous shape in computer graphics called the Stanford bunny. It was one of the original designs that represented a 3D shape in CAD, and it’s kind of become a running insert into every computer graphics paper.

Miller: What does it look like?

Lipton: It looks like a little bunny rabbit.

Miller: Is it easy to pick up the Stanford bunny?

Lipton: It’s very easy for a gripper to find a grasp that’s really secure. Imagine if you’re taking a bunny and you’re picking it up by its midsection by putting your hand around both sides. It finds a shape that locks in like that and can pull it up. And it could also find one that kind of scoops it from the back to hold it into position. And those were very secure grasps. We could tilt them almost 180° back and forth, and it was really locked in.

Miller: Okay, this is actually a helpful way to think about it in the last three minutes. Let’s say a company had been using a robotic assembly line to pick up coffee cups and to put them in boxes. Then they decide they want to make bunnies. Let’s say that this is commercialized. They call up a company that has your technology at 5pm, saying “we want to do bunnies by two days from now,” what would they do?

Lipton: All they have to do is give us the geometry in the form of CAD. We would process it. Within an hour or so, they have the shape they need to 3D print, and they could print it on site. And not only do we give them the shape of the gripper, we also give them the movement of the robot arm that will pick up the object.

Miller: And then they unscrew the cup holding gripper from their robots, screw in the new bunny gripping hand and they’re off?

Lipton: And they’re off.

Miller: How long would it be before that the technology you’ve just described is actually commercially viable and available?

Lipton: I’d love for it to be as soon as humanly possible. So if there’s anyone looking to commercialize this technology, please reach out. Right now, what we need to do is throw more objects at it to find where it will really break, as well as find more unique cases where there’s a challenge. So right now, we just need more and more data and real world experience with industrial deployments to really see what it would take to scale this up from picking up 20 objects, to 2000, to 2 million objects a day.

Contact “Think Out Loud®”

If you’d like to comment on any of the topics in this show, or suggest a topic of your own, please get in touch with us on Facebook or Twitter, send an email to thinkoutloud@opb.org, or you can leave a voicemail for us at 503-293-1983. The call-in phone number during the noon hour is 888-665-5865.