Impulse / Response is a 2-screen video installation where an object detection AI is invited to interpret physical movements and expressions of the body.

The technology that is typically used for surveillance is repurposed for poetic purposes. Every gesture is read by the AI system, which attempts to interpret it as an object. A succession of interpretations reveals a deeply practical, yet, probabilistic nature of AI.

Their sequence builds a narrative that reads like something between a dadaist poem and a shopping list — manifesting an interesting dichotomy present in modern technology. Built into the capitalist reality, it has to perform and deliver. Its algorithms are based on a constant fight between what is really there and what is expected (reducing the “loss function”).

Yet, the very working of those systems is aleatory and uncertain in nature, relying on a metaphor that is derived from a complex organic and analog world. There is an impulse, and there will be a response, but poetry arises in the moments of errors, misinterpretations, and silences where the system temporarily loses its function and has a moment of existence on its own.

…

The installation can be comprised of one or two screens and can operate from recorded material or live (using a custom-tweaked neural network engine running on Jetson Nano computer).

One screen is showing a recording or a live stream captured from a CCTV camera that is processed live by OpenCV algorithms into contours, which are widely used by object recognition AI for interpretations. Effectively, it shows how reality is seen by machines: a succession of differences from one contour to another.

In a two-screen installation, a second screen is adding the fractal variability control work that is reading and detecting the first system in real-time. This way, the surveillance AI is being surveilled itself in an attempt to make it more human and natural. Every time it deviates from the “natural” dynamics (identified by the observer network), it will receive an impulse, which will make it adjust its interpretation logic, establishing a feedback loop that is reminiscent of GAN (generative adversarial network) architecture.