art + math

The interplay among math, science and art may not be clear to many, but for Ellen Gethner, associate professor of computer science and engineering, finding ways to use math and science to create artwork is a passion. Inspired by a demonstration in her high school geometry class, she has been applying mathematical foundations to artwork for decades—from algorithms inspired by M.C. Escher’s mathematical work to translating music to a computerized visual interpretation.

“My geometry teacher colored a small square piece of paper with an M.C. Escher-like drawing and then he took four small, square mirrors, and stood them upright and perpendicular to one another around the drawing,” Gethner explains. “When looking at the reflection of the mirrors playing off of one another, you get an instant infinite and beautiful tiling of the plane, also known as a wallpaper pattern. That simple example combines math and physics to produce a stunning piece of artwork.”

The mathematics of M.C. Escher: Producing the infinite from the finite

Stemming from that high school geometry demonstration, Gethner has great interest in the mathematics of M.C. Escher’s work. It’s said that Escher worked to understand many mathematical principles only to use them in his artwork. One such project involved creating a pattern inside of a square tile that could then be rotated and reflected to create an infinite wallpaper pattern from that single tile. According to Gethner, his idea was to overlay a number of polygons in a square, call the resulting pattern the motif, make four copies of the motif, and then add four copies of the motif to a larger square made up of four squares in a grid.

“Before placing each of the motifs in the larger two-by-two grid, you are allowed to rotate and/or reflect each one,” she explains. “By doing so, you can create many ‘different’ square tiles. If you then tile the plane with this single square tile, you have a large variety of infinite wallpaper patterns made from one motif.”

Escher had an application for his initial tiling question—he wanted the ability to produce many different wallpaper patterns from either one (rotations only) or two (rotations and reflections) templates, so he was looking for a cost effective way to manufacture many patterns as well as give the consumer some say in the design.

The tricky part was to determine the number of patterns, which have come to be known as “ribbons.” Escher did a laborious by-hand investigation to try to count the number of different patterns arising from a two-by-two tile and, according to Gethner, he came pretty close to the right answer.

“The combination of computer science and math are central to the role of automation and efficiency this work requires,” she says. “I wrote several papers that gave the mathematics behind and a formula for the exact number of wallpaper patterns arising from using a single motif (rotated and/or reflected) n² times in an n-by-n grid square.”

Through her research, Gethner has also shown—using algorithms, graph theory and number theory—that upon input of a given motif in a square tile, one can always find a finite colored “prototile” that, upon vertical and horizontal translations, yields a colored wallpaper pattern in which ribbons are colored uniformly and overlapping ribbons are colored differently.

“This again gave rise to a method, albeit much more complicated, for producing the infinite from the finite. It was an exciting discovery and nice interweaving of different areas of mathematics and computer science that led to the solution,” she says. “With the algorithmic solution in hand, we wanted to use it to color some of Escher’s interesting motifs as well as design our own.”

What she has found, however, is that any pattern that requires five or more colors is difficult to make visually appealing without some depth of knowledge of what it takes to make attractive color combinations. This challenge led to some of her current work of making art with music.

“While some of this research may be seen to be subjective, there are scientific properties of sound and light that are of interest and of help to us,” she says.

Creating art from music

Using the natural numerical encoding of both sound and color, Gethner is working with Shannon Steinmetz, a master’s student in the integrated sciences program at CU Denver, to create artwork from music. The goal of the project, according to Steinmetz, “… is to construct a mathematical model that can provide an analytic mapping between repeated harmonic and/or dissonant melodies, and patterns of visually pleasing or displeasing color.” In other words, they’re working to find a way to convey the mood of the music through art.

So, how does one determine the numerical encoding of color and sound? This question is the heart of the research project, and once answered, the solution can be applied to other areas of Gethner’s research.

“We are learning about frequency of both sound and colors and working toward a usable method to match and then display them,” she says. “We have a way to go on this part and are both working to learn more about sound and color.”

For their first attempt, Steinmetz and Gethner are encoding sound and color using the frequency, which is challenging. Determining frequency of individual musical notes involves being able to sample the sound in real time and then converting it to artwork to be displayed on a computer screen.

Steinmetz devised a method to retrieve sound during a live performance and then convert it by way of a discrete Fourier transform—a mathematical function that transforms signals between time domain and frequency domain—to a sine wave. On said wave, the higher the amplitude the louder the note and the shorter the period the higher the pitch. As an experiment, using the time domain, Steinmetz created a visualization in which each note played is represented as an arc traced over a swath of three-dimensional space. Each arc has a starting and ending angle based on the changes in amplitude, and a radius that is dependent on the amplitude of the main signal. The resulting visualization is a set of geometric shapes that collaborate to form conic slices describing an interval of the signal. Colors are constructed by converting the signal information into red, green and blue values using amplitude, previous amplitude and signal-to-noise ratio, respectively. Ultimately, Gethner and Steinmetz want to leverage the discrete Fourier transform and frequency mappings to color a shape according to the frequency of its sound; all of these factors will play into the final artistic creation.

“The idea is that harmonious sounds should look good on the screen, and dissonance should look bad,” says Gethner. “A lot of this is very subjective, as it should be since we are trying to create art. The point is to be imaginative in transforming from one domain to another: The sky is the limit.”

To better help with this project, Gethner is taking piano lessons, through which she discovered an interesting puzzle. A C-major scale has a somewhat positive cheery sound while an A-minor scale (or any minor scale) is a bit sad and contemplative. The exact same set of notes is played in increasing order in both scales; the only difference is the note from which it begins. “Since our ultimate goal is to create artwork that reflects the music being played, we need to be able to detect the difference between major and minor,” she says.

“One of the very fun aspects of this project for me is that unlike proving mathematical theorems, which are either true or false, is that opinion and experiment play a big role in the outcome of how the result is automated by way of an algorithm,” says Gethner. “This kind of work has much more of an artistic flavor.”