Operational Images

‘In retrospect, perhaps I was meant to be baffled by the images in Eye/Machine III because, in fact, they’re baffling to human eyes. Farocki was trying to learn how to see like a machine.’[1] Farocki himself refers to the industry films and war propaganda of so-called ‘smart’ bombs as ‘operational images’: ‘The third part of the Eye-Machine cycle structures the material around the concept of the operational image. These are images which do not portray a process but are themselves part of a process. As early as the eighties, cruise missiles used a stored image of a real landscape then took an actual image during flight, and the software compared the two images. [...] Many operational images show coloured guidance lines, intended to portray the work of recognition. The lines tell us emphatically what is all important in these images, and just as emphatically what is of no importance at all. Superfluous reality is denied – a constant denial provoking opposition.’[2] It is clear that the machine no longer needs these digitally stamped in arrows, reference lines and coordinates. They are actually there to enable the human observer to read and interpret these images.[3] ‘In retrospect, there’s a kind of irony in Farocki’s Eye/Machine. Farocki’s film is not actually a film composed of operational images. It’s a film composed of operational images that have been configured by machines to be interpretable by humans. Machines don’t need funny animated yellow arrows and green boxes in grainy video footage to calculate trajectories or recognize moving bodies and objects. Those marks are for the benefit of humans – they’re meant to show humans how a machine is seeing.’[4]

Operational images are ‘images [...] which are not cut and framed to compress space and time [...] pictures [...] which as a rule are not perused by human eyes because they have been recorded to monitor a process. They are seen as so insignificant that they are erased and the recording medium is reused. Only in exceptional cases are the images perused and archived. Such images [...] have a beauty that is not calculated. In 1991 the leadership of the US military surpassed everyone in the art of showing something that approaches the unconsciously visible.’[5] This beauty is neither intended nor calculated but rather the result of a surveillance camera and the processing of what it records. In Erkennen und Verfolgen / War at a Distance Farocki shows how a special camera is directed at a steel strip in order to reveal breaks, tears and other faults, which are translated into the spectrum visible to the human eye: ‘The E-Ray camera captures what no worker could see, and image processing makes it visible.’ Rather than merely reporting on a process, the images are part of a process. ‘They are information and not really images. […] The working of their perception is made visible.’[6]

Algorithmic Guardrails

Before machines got underway, they needed algorithmic guardrails. The film-essayist Harun Farocki writes about the initial phase of machine-navigation using maps – not without scepticism: ‘The automatic eye has memorised a handful of filters, through which it scans the images of the real world. These image-processing apparatuses work with a clumsiness reminiscent of robot arms undertaking a new task. Every movement is divided into sections, with a pause after every partial movement. It is precise but bereft of elegance formed by habit.’[7] Farocki repeatedly returns to the aesthetic components of the operational machine-image, to which –precisely because it exists to carry out an operation – he devotes more interest than to the image that flaunts its value as artistic entertainment: ‘If we are interested in images that are part of an operation, this more likely comes from the flood of non-operational images, from the tedium of meta-language. The tedium of the everyday practice of re-mythologising everyday life and the multiple and multi-channelled programme of images designed to mean something to us. [...] The movie and television industry has worn out its material through overproduction.’[8] Due to their lack of expressiveness and form, he thus finds operational images rather exciting.

Farocki watches machines abolish human image production through digitisation after the end of manual labour, which was replaced by industrialisation: ‘Just as mechanical robots were initially patterned on manual factory labourers, soon outdoing and ultimately completely replacing them, sensory automatons are designed to replace the work of the human eye.’[9] EU banknotes have multiple markings which can in part only be read by machines. Other features are visible to the human eye or can be detected with one’s fingertips. All this will be unnecessary with digital currencies ranging from Bitcoin to the eEuro. Hand and eye of the workers still function as a training models, but the result ‘quickly leaves the model far behind’. To ‘fit in with digital methods’ manual work and the industrial past up to now must be ‘painstakingly adopted and translated into digital form’.

Twilight of Representation

Writer Geoff Manaugh has speculated about ‘how we could design spatial environments deliberately to deceive, misdirect, or otherwise baffle these sorts of semi-autonomous machines.’ The idea that in the future public infrastructure could be rebuilt to increase the efficiency of machine perception surely suggests the possibility of an environment ‘made deliberately illegible to robots […] with materials used for throwing off 3D cameras or LiDAR, either through excess reflectivity or unexpected light-absorption.’ What would be the capacity of ‘anti-robot architectural design, or anti-robot urban planning, and how could it be strategically deployed as a defensive tactic in war?’ ‘Smart’ cities as well as ‘smart’ homes would have to be designed to constantly confuse machine vision. ‘You thus rebuild the world using light-absorbing fabrics and reflective ornament, installing projections and mirrors, screens and smoke. Or “stealth objects” and radar-baffling architectural geometries.’[10]

Artists like Harun Farocki and Trevor Paglen have also reflected on critical approaches to operational images and spaces. While the current rediscovery of eastern German photographers has given rise to a retrograde hope that the indexical seen document can be trusted, others are working on the development of highly artificial worlds. The classical essay film no longer depicts workplaces or street fights. Instead it translates its analysis and critique into colourful, boisterous, computer-generated and constructed spaces.[11]

In the Kunstsammlung Nordrhein-Westphalen art collection in Düsseldorf, three parallel individual exhibitions of works by Hito Steyerl, Thomas Ruff and Simon Denny recently illustrated the extent to which camera-based art has been supplanted by algorithmic image production. The photographer Thomas Ruff has put aside his camera and now only works on digital formats found on the internet; the video artist Hito Steyerl works in an area that combines gaming, chats and Youtube; and the installation artist Simon Denny constructs digital gaming zones and computer-animated videos and theme-park displays.

Mine

Originally taking the mining regions of Australia as his reference point, Denny subsequently transferred his work Mine to the Rhine/Ruhr industrial region. The K21 museum in North-Rhine Westphalia, where the exhibition was held, is located not far from the remains of the contested Hambach Forest, which has been heavily impacted by brown-coal mining. But Denny chose to focus on the Zollverein Coal Mine Industrial Complex in Essen, which has been converted into a museum. Working together with artist Jan Berger, he connected a digital journey through the mining museum with the K21 museum using the game world Minecraft. The exhibition thus took place both in the cellar of the virtual museum complex and in the real exhibition space.[12] However, the computer graphics displayed on the wall and floor and the cardboard stand-ups of cyberpunk-like mining products rather recalled shabby trade-fair architectures or theme parks.

Mine thematically interwove the extraction of raw materials and natural-seeming data and tracked the effects on human labour power. The title referred not only to mining but also to the possessive pronoun, thereby interweaving the overexploitation of habitats and human beings – as seen in the presentation of a smartwatch by the mining equipment producer Caterpillar, which promises to ‘predict, measure and reduce risks of fatigue and distraction’. The device records the ‘sleeping and waking periods of the staff member and translates this data into an effectiveness score’. Like self-driving drilling machines, trucks and trains, the remaining workers are digitally monitored from another location and their data is processed. These pit control centres remind us of the drone control centres investigated by Harun Farocki as part of his work on ‘operational images’.[13]

Cage People

The crazy thing about Denny’s exaggerated product videos displayed between cardboard stand-ups and court drawings of fictive lawsuits created by Sharon Gordon is that they are based on existing advertising material and are market-ready. Currently small robots are being deployed to assist human workers. These machines are no longer dangerous to humans and therefore do not have to be kept in protection cages. Previously people and machines had to be kept apart and protected from one another. In Erkennen und Verfolgen / War at a Distance Farocki shows how after mounting car parts on welding and assembly robots the workers still have to lower a gate and step back.

Recently Amazon presented US patent 20150066283 A1, a drivable cage with an external gripper arm in which a so-called picker is supposed to work. By building a version of this patented ‘Amazon worker cage’ Denny brings remote activities into the framework of the museum and allows us to zoom in on them directly.[14] A digitally animated Thornbill bird tweets from the cage; its extinction, which may have already happened, is a warning of the deadly effects of worldwide CO2 emissions – a parallel to the historical use of canaries in coal mines as a kind of early warning system for the detection of dangerous carbon monoxide.

The planetary supply chains and ‘victim zones’ of the Capitalocene designate the Earth, people and even museums as sites of the exploitation of raw materials and data. What insights are offered by Denny’s ‘gamification’ of mining – he goes so far as to offer the board game Extractor in lieu of a catalogue, with the game instructions also functioning as a catalogue brochure? Mechanical and digital automation will even take over the work of raw materials extraction – bad news for the African continent, where late industrialisation offers hope of approaching the general level of prosperity of the Global North or China. Before production from Asia overflows further onto the African continent, as we are already seeing in Ethiopia, Morocco and South Africa, machines there will probably already be faster. People will thus lose their jobs before they have even gained them.

The Will to Enlightenment appears in the Garments of Gaming

‘In 1991, it was often stated that images of approaching targets and detonations captured by surveillance cameras made the war look ‘like a computer game’. [...] Nearly all technical representations that claim only to present the functional principle of a process involve a high degree of magic.’ Farocki evokes an image of war machines continuing to fight without people: ‘The idea of a deserted battlefield on which war battles on – a bit like the toys that come to life when the children are sleeping – reminds us of the emptiness of production facilities. In the automobile industry, for example, we only see people working where there is no room for another robot.’[15] But where are the people and the cities in this scenario? In Farocki’s Erkennen und Verfolgen / War at a Distance we only encounter urban life when, in a curious montage, the director couples the view from a panzer hatch with a digital window-shopping tour along a street.

The film repeatedly shows a guided missile flying over a digital, starkly abstracted landscape, which recalls a camouflage pattern more than modulated spatial sequences. ‘The missile and the simulated flight is practicing navigation, using a synthetic landscape image. These synthetic images have been purged of anything that could be a direct clue to human life. Fields, roads, villages: all have been reduced to a symbolic level.’[16] For training purposes, images of the landscape that is to be flown over are shown to a mechanical eye. ‘These are analogue images taken during a real flyover. […] The seeker head goes through these flyover images and processes them.’[17]Only just before detonation we see a street intersection with recognisable houses and a vehicle, which is hit by the missile and bursts into flames. The details of a machine-readable city and its completely invisible inhabitants are only of interest here as a target at the moment of their destruction.[18]Mathematical eyes look around, see patterns, compose, within their conceptions. It all comes together.