The gripping story – which became one of the biggest art scandals of the 20th century – began back in 1894, when the Imperial and Royal Ministry of Education announced a commission to decorate the Great Hall of the new University of Vienna on the Ringstrasse. After being put out to tender, the contract was awarded to the artists Gustav Klimt and Franz Matsch. It promised to be a lucrative commission for the pair as it comprised five large ceiling paintings and ten smaller works with academic themes. The brief called for one representation for each of the four major faculties at the university: namely theology, philosophy, medicine and jurisprudence. At its center, an image entitled The Victory of Light was planned.

When Franz Matsch unveiled his first sketches for the eight-meter-long central image, responses were favorable across the board. Klimt also presented the design for one of a total of three paintings he was commissioned to create. And his design likewise was met with approval. So they set to work, spending several years on the paintings. But these were years in which Klimt’s style was to change dramatically. The die for a huge controversy had been cast. And the art scandal soon gathered pace.

Show all pictures

1 /of 3

3 pictures

The Unadorned Truth

When Klimt presented his three finished paintings to the clients, it triggered a huge wave of outrage. His pessimistic – and critical – take on science was met with incredulity. Because, rather than celebrating it as custom dictated, Klimt's Faculty Paintings showed the unadorned truth of illness and poverty. A radical approach. The client’s criticism was that Klimt's works failed to dovetail with the overarching concept. Huge controversy followed – and his Faculty Paintings were never displayed in the Great Hall. But that’s not all: bitterly angry, Klimt bought the paintings back with his own money and vowed never to accept another public commission again. Instead, he channeled all of his energy into his portraits and landscapes, cementing his reputation as a revolutionary artist and going on to achieve international acclaim. But the paintings themselves were not even put to rest even after Gustav Klimt's death in 1918. Under the Nazi regime, the works – then part of the famous Lederer Collection – were seized and Aryanized. The curtain finally came down on the Faculty Paintings in May 1945, by which point they were in storage at Schloss Immendorf, a good 50 kilometers to the north of Vienna. In the chaos of the last days of the Second World War, the Nazis set fire to the castle. Immendorf went up in flames along with the Faculty Paintings and many other important works of art. It was bad enough that the originals were lost: but there weren’t even any color photos of the Faculty Paintings. Only a handful of black-and-white photographs taken for posterity captured what Klimt had created. The end of the story? Almost.

Putting the Art into Artificial Intelligence

Almost 75 years after the paintings were lost, an AI puzzle hunt began to solve the artistic mystery. Österreichische Galerie Belvedere, home of the world’s largest Klimt collection and one of the city’s top museums, joined forces with Google’s Arts and Culture Department to create a digital platform by the name of Klimt versus Klimt. In all, the project involved 33 partner institutions from twelve countries. One element of the project centered on an attempt to bring Klimt's Faculty Paintings back to life using a special algorithm designed to give modern-day beholders a feel for the original coloration of the paintings. A spectacular project that united international IT specialists, art historians and Klimt experts.

When Data gets Creative

The colorization process applied in this particular case was based on an algorithm that had actually been developed to colorize black-and-white photos. It was then adapted and fine-tuned specifically for the Klimt project. A highly complex undertaking that also incorporated elements of machine learning – a branch of artificial intelligence that draws on sophisticated statistical models. Or, put another way: high-performance computers were fed with large volumes of data, which, when taken together, delivered clues about coloration in the art world at that time. But for the algorithm to produce reliable results, thousands of comparator images from this period of art were required, as well as additional algorithms which were interlaced with each other. In total, the project drew on information taken from around 90,000 works of art. At the heart of this data was a set of around 80 paintings by Klimt that the Belvedere's experts deemed as being of particular relevance for the reconstruction of the Faculty Paintings, as they either hailed from the same creative period or had parallels with the controversial works in question in terms of the motifs they depicted. But the Google Arts & Culture and Belvedere team were not only involved in painstakingly processing direct visual information.

Unexpected Outcomes

As it turned out, the extensive research work carried out in libraries – and in newspaper archives in particular – played an equally important role. To the surprise of all involved, these contemporary sources proved to be a rich vein of information, as the art critics of the time wrote astonishingly precise descriptions of the three Faculty Paintings in their exhibition reviews. As a direct result of these accounts alone, more than half of the motifs could be assigned very specific colors. It proved a key step in the reconstruction of the images as it transpired that Klimt had very closely oriented the final pieces to the colors featured in the first drafts for the paintings, which was by no means a given. And the researchers had a major stroke of luck when it came to the Faculty Painting dedicated to medicine: a color photograph of a small section of this painting appeared in a 1931 publication on Gustav Klimt, which provided further vital clues regarding its final appearance. Piece by piece, the puzzle was starting to come together. After almost four years of work, the time had come: Klimt's Faculty Paintings were back to their original technicolor best, in a spectacular reconstruction showing just how stellar the art of Viennese modernism was. And another fascinating insight also came to light for art historians: the reconstructions show very clearly that Klimt's Faculty Paintings were not just out of line with the others in terms of style. The final color palette he used was completely different to the one featured in Franz Matsch's contributions. From this perspective, the clients were well within their rights to reject the images. But for the art world, it was a fateful decision, because the paintings would have survived the Second World War unscathed if they had been displayed on the ceiling of the University of Vienna.

Text: Johannes Luxner

© WienTourismus/Paul Bauer