ADDITIONAL MATERIAL TO SUPPORT DIGITAL DEAL SUBMISSION
20-minute documentation from three performative ethnographies carried out during the summer 2023, investigating the protein transition in Dutch agriculture. Produced by Artphy, Ede, the Netherlands
The central premise of the work Reading Lips was that the cognitive mistake (apophenia) of seeing mouths in stomata would occur both in humans as well as algorithms made by them. Indeed even early experiments with off-the-shelf applications showed successful ‘mouth’-detection (far left), which we followed up with LipNet Neural Network implementation; to get this algorithm to work, however, we had to change the colour of the stoma to match (white) skin tone. After that, LipNet had no trouble interpreting the leaf pore.
PL’AI, the AI-controlled robot made to enter plant time and be able to play with cucumber tendrils at plant speed, was a complex and challenging artwork because of the translations between living materiality and the abstract realm of the digital. The sensors are limited in their modalities and capacities, so we developed many tools that helped visualise what was going on in the “mind of the machine”, creating hybrid representations of video and simulation to assess what the robot was sensing and how it was making decisions. Left: side-by-side comparison of top view machine vision analysing tendrils touching the coloured balls. Right: a latent-space walk - a reanalysis of the matrix of the cummulative 3D LIDAR scans to travel along the axis of time. Below: excerpt of close-up time lapse of cucumbers playing with the robot during the exhibition.
This collaboration with Miha Turšič and the supercomputing center in Edinburgh was the result of the FEAT residency. The installation was an instrument that explored interoperability between AI and and various digital media, referring to various social, cultural and scientific dimensions. Starting with random input of zeros and ones, the instrument eventually produces narratives, seemingly from nothing, as an artefact of the procedural intentionality.
Full-length film from the installation AIxxNOSOGRAPHIES (2023); it is a reimaging of documentation from Performative Ethnographies carried out within the project AIxxNOSOGRAPHIES, through the lens of speculative normalisation.