Tate exhibition shows what happens when art and artificial intelligence collide

Tate Britain has launched a remarkable experiment in mass pattern recognition, in conjunction with the 2016 IK Prize for digital innovation in the art world. This year’s winning project, Recognition, was created by the Italian group Fabrica.

Peter Lely’s Two Ladies of the Lake Family, 1660. Courtesy Tate Britain

Great art should not be reduced to a computer algorithm. Who, after all, could codify, quantify and automate that ineffable power of one human being to reach deep into another’s soul and induce the stirring feelings of aesthetic beauty? “Painting is poetry that is seen rather than felt,” as Leonardo da Vinci put it, and cannot be reduced to the domain of the robots.

Or alternatively, maybe we should be a little less precious, and a little more curious. This week Tate Britain launched a remarkable experiment in mass pattern recognition, in conjunction with the 2016 IK Prize for digital innovation in the art world. This year’s winning project, “Recognition”, was created by the Italian group Fabrica, and uses several types of artificial intelligence algorithms to pair up classic works of art from the Tate collection to the latest pieces of photojournalism provided by Reuters.

Over the next three months, the computer program will create an ever-expanding virtual gallery, as more and more contemporary Reuters photos roll in – with the opportunity for visitors to the Tate’s real-world gallery in London to intervene in the selection process, and potentially improve the algorithms’ decision-making. Recognition looks for matches between paintings and photos in four different categories.

The first is “objects”: so the software records a tree with leaves, a man in a black hat, a clock, a red umbrella. The second, “faces”, where an image contains clearly visible portraits, seeks to put an age and gender on the face, and match accordingly – I’ve spotted this fail pretty badly. Augustus John Om’s painting of “A Jamaican girl”, is not a portrait of a male, aged 51. As the title suggests, it is a Jamaican girl. Score one against the robots. Bad robot. The third category, “composition”, is probably the most important of the four, in catching the eye, and showing us two images which actually bear a serious resemblance.

Finally the fourth, “context”, looks at metadata tags (ie words attached to the photo or painting) for connections: field, construction site, horse, table, outdoor, sky, and so on.

The Tate’s promotional slogan for the project – “connecting British art to the everyday” – is a bit of a misnomer; the suggestion is that the second half of the pairing, the photojournalism, is merely “everyday”: quotidian reality as we see it through our eyes, and at a remove from art itself.

Sure, it’s just a marketing slogan, and can be happily ignored– but it obscures why this project works: because the best photojournalism not only performs the perfunctory service of documenting unfolding news stories, but reveals the profound beauty to be discovered in the “everyday”.

It is heartening that in an age where much of the world has instant access to a camera, via digital cameras and smartphones, that there seems to be a growing popular understanding of the power of photography to do more than just create personal records – or build social capital on sites like Instagram – but to capture the innate beauty of the world around us.

In 2014, a remarkable photograph of a brawl between MPs in the Ukrainian parliament, taken from above, arms sprawling, was compared to a Caravaggio painting. This quickly turned into a popular crowd-sourced game for Twitter users, as members of the public found other pieces of photojournalism which displayed the “Fibonacci spiral” or “golden ratio”, the instinctively aesthetically pleasing geometric pattern found in nature – in the shape of a pine cone or a flower, for example – and beloved of Renaissance painters.

Working through the nascent gallery of matches on the Tate Recognition website, some of the juxtapositions are profound or comically bathetic because they are so unexpected – and because they defy the algorithm.

Two striking pastoral scenes are matched largely through the “composition” and “context” algorithms – one, Sir John Lavery’s 1922 oil on canvas painting of “The Golf Course, North Berwick” – depicts a few distant figures dwarfed by a far reaching grey-blue sky; the pacific sense of a post-World War One middle-class returned to their leisure. It is paired with Jorge Silva’s photo from a new series about the “lethal legacy of secret war in Laos”, and despite strong compositional similarities – a figure on an expanse of green grass, beneath a cloudy blueish sky – we sentient humans can see the figure is holding some kind of detecting device, and from context clues (and some web searching) discover that he is looking for unexploded ordnances: since the end of the Vietnam war, more than 20,000 Laotians have been killed or injured by bombs dropped by the US army.

The two images would never have been brought together but for the fact that a piece of AI thought that they looked similar; but who knows what fresh ideas might emerge from these unlikely robotic collisions?

• Recognition is on display at Tate Britain in London until November 27. To view the project online, go to recognition.tate.org.uk

Dan Hancox is a regular contributor to The Review.