About 75 years ago, Jackson Pollock revolutionized the art world with his distinctive style of painting. He would lay the canvas on the floor and with his arms outstretched, pour or drip cans of paint directly onto its surface. The technique invited admirers and detractors alike, along with scandals involving forged canvases turning up decades after his death.
Richard Taylor, a professor of physics, psychology, and art at University of Oregon has been using computers for more than 25 years to analyze Pollock’s paintings and help authenticate canvases of uncertain origin. He recently collaborated with two former UO doctoral students to develop a tool using AI to distinguish between genuine and imitation Pollock paintings with 99% accuracy. Taylor joins us to talk about the recently published results, and the role AI may increasingly play in the art world.
Note: The following transcript was created by a computer and edited by a volunteer.
Dave Miller: From the Gert Boyle Studio at OPB, this is Think Out Loud. I’m Dave Miller. More than 75 years ago, the painter Jackson Pollock dripped and poured his way into art history. His distinctive splatters invited admirers, and some detractors. In recent decades, when Pollock’s painting started to be sold for tens of millions of dollars or more, they also invited plenty of imitators. Now, an AI tool could be used to sort the real ones from the fakes. The project was spearheaded by Richard Taylor. He is a professor of physics, psychology, and art at the University of Oregon. He has been interested in Jackson Pollock for much of his life. Richard Taylor, welcome back to Think Out Loud.
Richard Taylor: Thank you, Dave.
Miller: Why did you want to use AI to identify Jackson Pollock imitations and Jackson Pollock real ones?
Taylor: Well, it’s almost the perfect storm. The Jackson Pollock paintings are unusual, they’re very abstract. So that kind of invites people to think that they can create a master work much like Pollock’s. You have that sort of idea that anyone can create a Pollock, and then you couple it with these astonishing price tags that you’ve just mentioned. The highest ones sold on the market is $140 million, but ones in museums that are not on the market are valued anywhere up to $400 million. You combine those two things, and that means that we have this idea, this situation where these master works are out there, but also there’s literally hundreds of fakes in circulation.
We want to make sure that that legacy of that great artist is protected. So what we don’t want is any of those fakes to actually be misunderstood as a master work.
Miller: You’ve been called in more than once to weigh in on forgery scandals or allegations of forgeries. What are those like?
Taylor: Haha! I can laugh about them now, but when you’re in the storm, they are no laughing matter. They often grab the media attention around the world, and there’s a lot of pressure for the reasons that I’ve been saying to actually get it right. So in retrospect, in terms of looking back and thinking, well, that’s great that I could use my knowledge of Jackson Pollock to help resolve an issue, it’s definitely rewarding. But at the time, it’s a very high pressure situation.
Miller: Let’s turn to how you actually went about creating this AI tool. You did this with two postdoc researchers at the U of O. So the first thing to do from what we’ve heard in the past, if you’re going to create this kind of algorithm, is you’ve got to feed a lot of data into it, both the real ones and the imitations, so that the machine learning can actually happen. How many real, verified Jackson Pollock paintings are there?
Taylor: Not that many – 189. So that was one of the challenges, that we had to assemble as big a collection as we possibly could. As you mentioned, I’ve been fascinated with Pollock for many, many years. And so gradually, I’ve been collecting collections of known imitations, and also of the authentic Pollocks. I think one of the special things about what we’ve done, because any sort of AI is only as strong as the images that you train it on, is that we managed to collect the biggest collection of Pollocks and imitations that there are.
Miller: My understanding is that because you had a limited number of verified Jackson Pollocks, you ended up essentially chopping up those digital images into smaller panels, to give a higher raw number of images to train the machine on. But I’ve been confused by that, because I don’t see how that’s more information than just giving the machine very high resolution full paintings? What am I missing?
Taylor: Yeah, it’s a great question. It was a very, I think, moment of genius. And I can say that because it wasn’t my idea, it was actually my collaborators’, Caleb Holt and Julian Smith. It’s all based on some previous computer analysis that we did, where we discovered that Pollock’s work is actually what’s called “fractal.” Fractal patterns are just patterns that repeat different magnifications. So what that means is, with a Pollock painting, is that you can look at it from a distance and look at all of those patterns. But if you zoom in on it, you get very similar patterns. So in a way, a Pollock is made up of all of these miniature Pollocks strung together.
Caleb and Julian realized this, and realized that they could lay down this tiling process – separate out all of these miniature Pollocks. And we did that both to the Pollocks and to the imitations. So suddenly we went from having a total of 600 paintings, to 250,000 tiles. And that adds considerable strength to the training of the AI machine.
Miller: Where did you find versions of the drip-style paintings that could serve as negative examples, as what we’d call imitations, or maybe fakes if they’re done on purpose?
Taylor: This was really fun, because we really grabbed a whole diverse set of images, ranging from some very well known ones. So many people might remember a great film on Pollock called “Pollock,” and Ed Harris starred as the actor. He took the role very very seriously, and he practiced and studied Pollock, and was actually well known within the art world [as], in a way, capturing Pollock’s painting style. So we actually went and used some of his paintings in the study. There are famous artists who’ve also tried their hand at Pollock’s work. Max Ernst is a great example, Hans Hofmann is another one. And so we grabbed some of those. Even a monkey has been celebrated for generating Pollock paintings.
We also did these Dripfests, where we invited adults and children to actually try their hand at Pollock paintings. So in the end, we had a huge variety of over 400 imitations that we could use.
Miller: And then what did you find? How accurate is this?
Taylor: Well, really good news, and better news than I was expecting. Because these controversies that we mentioned earlier have usually arisen because two prominent experts had differing views. One said that a newly discovered painting was by Pollock, and one said that it definitely wasn’t. So when you use the human eye, it’s not 100% accurate. Even the best trained people get it wrong. So it was a really fundamental question: to what extent could artificial processes actually study a Pollock, and could it do better than a human being? And that’s a really interesting question, because in a way a computer is not meant to be looking at art, it’s created to be appreciated by human beings. So can a computer do better?
We arrived at a really high accuracy. Its accuracy is 99% in terms of getting it right.
Miller: What stands out to you in that 1%? If anything, what characteristics do the false positives share?
Taylor: Well, they’re very rare. In fact, there’s just a couple of them. And those are very, very interesting. And it’s not that the machine got it wrong. It’s actually that these paintings did display the visual signatures of Pollock. One of them is by a French artist called Henri Michaux, who was actually part of the French equivalent movement to the American movement that Pollock belonged to. So it’s very reasonable to expect, had very similar artistic missions as Pollock, wanted to do very gestural paintings. And when you look at them, they are very similar.
The other one that got it right, I’m very proud of. Because we actually created a machine called the Pollockizer, deliberately to see if mechanically we could create an artwork that resembled Pollock’s paintings. And one of the Pollockizer images did actually manage to do that so well, that it kind of fooled the machine.
Miller: It does make me wonder if you found old linen, old wood for a stretcher, the kind of paint that would have been used in 1948 or something, and you used your Pollockizer machine, do you think you could create a painting that would fool not just humans but also your AI?
Taylor: And that’s literally the multimillion dollar question. Because people do exactly what you’re saying. They take old canvases, scrape off the pigments, use old canvases so that they’ve got all of the materials that chronologically make sense and then repaint. So the question is, could we use the Pollockizer to do that? And that’s the battle that we’re in. Because we’ve now used AI to detect paintings. But as many as the listeners will know, you can actually use AI to generate paintings. So perhaps we’re heading for this sort of fascinating war, where the good guys like me are using AI to detect the forgeries, and the bad guys are actually using AI to generate forgeries.
Miller: Do you know why the tool that you helped create is accurate? Do you know what it’s doing? Or is this a classic AI black box where you know what you fed it, but you don’t know what it learned?
Taylor: One of the strengths of this AI technique … because it is truly artificial intelligence, it’s modeled on the human brain, it’s an artificial neural network. So like you say, it’s a black box that we feed in the images, but it goes away and works it out. We don’t inform it in any way. And that’s beautiful for us in terms of the art world, because previously a lot of these controversies have been fueled by this concern that judgments are subjective, and perhaps even might be motivated by some sort of negative desires to fling decisions in particular directions. So the black box is very important, in that we can actually say we have not influenced this decision. This is the decision of the black box.
But because it’s a black box, you have to be very careful that there’s no madness going on in that black box. So we have done a number of tests to make sure that what’s coming out makes sense. I mentioned earlier on, this idea that Pollock’s work is fractal. We have actually gone away and done fractal analysis on the artworks, and shown that there’s a big correlation between the fractal character of the artwork, and when the machine is actually declaring that it looks like a Pollock. So one of the rewarding things for me then is that we’re actually getting an insight into Pollock himself, that this isn’t just a technical authenticity tool, but we’re learning something very fundamental about Pollock himself.
So we do know that when the machine makes a decision, it is heavily informed by the fact that it’s looking for these fractal patterns in Pollock’s work.
Miller: Is there something about Pollock’s work that lends itself to this kind of computational analysis? I guess what I’m wondering is if you fed this AI of 180 Rembrandts or 300 pieces of Van Gogh paintings, as well as the same number of imitations, do you think that you’d come up with something as effective?
Taylor: This is a great question because in the AI world, this is known as generalization. If you’ve trained it on one thing, how well does it do on another?
We did put this to the test with these paintings I mentioned, what we were calling the Dripfest where we got a bunch of kids to actually generate the paintings. And so then we turned the question from “can you detect a Pollock out of all of this group” to “can you detect these kids’ paintings?” And it did extremely well, the accuracy was also very high up there.
So we do know that it generalizes to poured paintings. However, then your broader question, how well would it be do on other artworks, we’ve not tested that yet. I suspect that it will do very well on abstract, very gestural paintings like Pollock’s. But I think when you start to think about more illustrative artworks, like artworks that feature faces or human figures, I think it will actually do less well.
Miller: How do you imagine that this particular tool could be used?
Taylor: We are in a world now where AI is becoming more accepted. For example, with our machine, we wanted to try and create a machine that would be readily accepted by the art world and wouldn’t be controversial. Our machine is already being used in society by other people on quite critical things such as medical applications, cancer detection, through to security and antiterrorism at airports. So the type of neural network that we’re using has been applied in many different functions across society already. Now, we’re just taking this AI idea to the art world.
I’m hoping that it will be embraced, and that it will be just an extra tool to add to all of the already existing tools to help stop all of these scandals that keep on regularly appearing in the art world.
Miller: I don’t suppose the people who already own or bought paintings that they have been told are Pollock’s would want to get anywhere close to your AI?
Taylor: That’s right. It’s always a dilemma with these things about how much do you want to know? If you’re fundamentally interested in the artwork, in terms of knowing everything about it, of course you want to apply the AI technique.
Miller: And if you’re the potential buyer too.
Taylor: Yeah, that’s right. So I think it’s more to help inform people who are about to make this massive investment into something.
Miller: I’m just curious, you have been captivated by Jackson Pollock for many years now. Why? What is it about Pollock that has maintained your interests since you were in your twenties?
Taylor: Even before that, Dave, I remember as a little kid, maybe nine years old, a teacher brought in a box of donated books and I sifted through them. One of them was an art book and it had this painting by Pollock in it. And even as a nine year old, [it] just mesmerized me. It’s a very human thing where you hear your favorite piece of music or something, it just made me happy looking at them. And that’s good enough. Often, you don’t need to know what makes you happy, it just does. But even at that early age, I was not just mesmerized by it, but very curious about what is it about a Pollock painting that I found really striking, in a way that a lot of other things I didn’t? And that has really informed my career quite powerfully, this idea that can you use science to actually understand art? And so it all started as that little nine year old kid, and has ended up with what we’re talking about today.
Miller: Can you imagine your career without including art in it? Having it be more classically what people think about as “purely scientific?”
Taylor: I can imagine it, and that imagination comes up with a picture that’s pretty miserable. I’m very thankful for the fact that I’ve had opportunities to combine lots of interests. But I think that that’s fundamental. I think that many people have lots of diverse interests. And if we can encourage them to combine those interests, I think it’s a very good thing.
Miller: And this is not the first time you’ve used computers to understand Jackson Pollock paintings, your fractal work goes back a number of decades now. But has this AI research changed the way you think about these paintings that you already know very well?
Taylor: The thing that really I think is interesting is like you say, we’ve used computers to actually detect patterns on artworks. And that was kind of controversial when we first announced that, because a lot of people said “But these artworks are not meant to be seen by computers. Why are you doing this?” And then today we’re doing even more. We’re actually getting a computer, an artificial system to think about an artwork. It’s a great question. Does that mean that this AI tool is actually appreciating the artwork? Because what we’ve shown is that this AI machine can actually distinguish a real Pollock from an imitation more accurately than a human being. So does that mean that the computer in some way is actually appreciating that artwork?
Miller: Can I answer that question? Not to me. It doesn’t mean that necessarily at all. It means that, computationally, it sees patterns and it can detect them.
Do you answer that question differently? I know you post it as a kind of hypothetical, but as one person talking to you, I have a ready answer for that hypothetical.
Taylor: And what I find interesting is this sort of parallel with the human visual system, because that’s my sort of day job, if you like, studying how human vision works. And it’s an extraordinary system. But there are aesthetic models that tie what you like to look at with how easily can you process that information. When you look out at nature and you find that scene beautiful, a lot of our research is showing is because your human visual system through evolution has learnt to process it very efficiently. So there are models out there that are saying that processing efficiency does trigger an aesthetic experience.
Miller: Richard Taylor, I look forward to talking again. Thanks so much.
Taylor: Thank you.
Miller: Richard Taylor is a professor of physics, psychology and art at the University of Oregon.
Contact “Think Out Loud®”
If you’d like to comment on any of the topics in this show or suggest a topic of your own, please get in touch with us on Facebook, send an email to thinkoutloud@opb.org, or you can leave a voicemail for us at 503-293-1983. The call-in phone number during the noon hour is 888-665-5865.