Seurat, Robotic Painter

This web page outlines work we did in 2007 - 2008 trying to get machines to express themselves artistically. We develop C++ code that has three major components: 1) a physical paint simulation 2) algorithms that try to make paint strokes look like a target image given the constraints of physical paint properties and 3) code that turned strokes into machine instructions for a robot to execute.

By the end of the journey Seurat had painted a handful of paintings and had produced various original styles.

reverse engineering an image:

The first painting algorithm was designed around my intuition about how we could most successfully represent an image with a robotic painter. I would use three different brush sizes and the robot would start with broad strokes and would be given smaller strokes to fill in smaller details. The algorithm decomposed and reverse engineered a bit map image into channels and features that brushes could execute. Although we got good response from this video, there was something unsatisfing about the process.

two early painted images:

making machines creative:

The next step was to try to make the system as creative as possible. But really, how originally creative can a computer / robotic system be? The answer is that we are nowhere near a stage in artificial intelligence where a machine is going to go choose its subject and medium and create an original work of art. But in certain domains machines and computers do have the ability to be creative and come up with 'ingenuous' and interesting ideas. What types of problems can computers solve creatively? Very low level problems: like trying to represent an image physically. Given a certain objective: paint an accurate picture, and sever physical constraints, the robot would be able to come up with a creative way to try to express the source image in an original style.

Four evolved painted simulations. Walter Benjamin in 100 straight line strokes.

In a way the creativity of the machine and the person are intertwined: the person is making high level decisions about what image to use and low level decisions about brush sizes and pigment colors. After using the system for many months I can say that I was truely surprised by the look and feel and sense of style that the system came up with given my inputs into the program.

videos:

First algorithm in action
Evolving paintings in simulation

briefly on methods:

In order to let the robot come up with it's own style, we had to give him a way to judge himself. We created a painting simulation based on the Kubelka Munk theory of reflectance. The study of paint that we did was pretty advanced for the scope of work, but could be useful in future intellgient painting systems. The system also used genetic algorithms to represent and explore the space of posssible paintings. More information can be found in the pdf in the links seciton.

proof of concept evolution of simple image

future works:

While we did a lot of work on paint simulation, the simulation only matched reality for low amounts of in canvas blending. Some of the most compelling simulated paintings were the result of allowing significant in canvas blending. Devoloping the paint simulation to a level where we could model this in canvas blending more accurately would allow us to physically paint some of these images.

Another possible avenue of work could use realtime feedback, visual or physical to adapt as the robot executes the painting to allow the system to take advantage of all of the complex interactions that happen in reality. This visual feedback could allow the system to improve its self model and evolve its painting styles over time.

In our increasingly digitized world, I see a lot of opportunity for people to colaborate with machines creatively in traditional mediums.

Evolving a painting with many strokes with little paint mixing

from the paper:

Why paint?

Even the technological advancements of the post-modern age cannot replace the simplicity or cultural significance of paint and other traditional media. Artists have created representational art from suspended pigment for tens of thousands of years. From cave paintings through the Renaissance to the modern and contemporary eras, paint has never ceased to have a significant role in art. Traditional mediums will always have a power associated with the history of art: every new painted artwork refers implicitly to the history of painted art.

We have developed a robotic system which allows a computer to generate a physical painting from a source image with the aid of the technartist's artistic interpretation. Given an input image and environmental parameters, such as brush sizes, the combined computer-artist system is able to interpret the image and the robot then executes the image onto a canvas using paint.

Why Genetic Algorithms?

Genetic algorithms are stochastic algorithms able to solve some open-ended problems in creative and ingenious ways. These algorithms are especially useful when a solution of a problem can be accurately modelled and easily evaluated. GA's can often free a computer from human imposed limitations on a problem and can achieve unexpected results. For these reasons artists have found GA's powerful when trying to create generative art.

Problem Statement in Engineering Terms

Our goal is to take a digital image and have a computer analyze it and produce a physical representation of this image in an original style. This problem is fundamentally different than the problem of painterly rendering in computer graphics.

In computer graphics, arbitrary paint behaviours, paint colors and stroke types can be used to achieve whatever type of affect the user or the algorithm requires. Our research however addresses a physical problem: what instructions can I give a robot to produce a painting? Real paint phenomena must be recreated and predicted in a computer and the result of the algorithm is a set of robotic instructions, not a digital image.

Our work is also materially different than other software based art that can be outputted using a printer. While many software systems can claim original styles of art, these styles are fundamentally digital and can only be transcribed to reality using digital processes which utilize the same digital primitives that a computer uses. One of our central claims is the ability of our system to use the physics of the resulting art in its creative process.