Lilia Perez Romero, Frontera v.2

Artist in Residence april – juli 2008

Na een verblijf van drie maanden als Artist in Residence in het Nederlands Instituut voor Mediakunst presenteerde de Mexicaanse kunstenares Lilia Perez Romero het werk Frontera v.2. In deze interactieve installatie reageren digitale portretten op de bewegingen van de bezoeker.

Eerder ontwikkelde Perez Romero in het Center for Multimedia in Mexico Frontera v.1, een veel tentoongesteld interactief portret. Tijdens de drie maanden bij het NIMK verbeterde en ontwikkelde Perez Romero dit concept verder met behulp van een open source toolkit dat tevens door andere kunstenaars kon worden gebruikt. Daarnaast voegde zij een videohokje toe aan Frontera v.2, waar bezoekers nu zelf ter plaatse een digitaal portret konden maken en dit konden toevoegen aan de database. Hierdoor kon de bezoeker spelen met zijn eigen geprojecteerde spiegelbeeld, dat reageerde op zijn bewegingen.

Met Frontera v.2 blaast Perez Romero met nieuwe technologieën het traditionele portret nieuw leven in. Perez Romero: “Dit is een logische stap in de ontwikkeling van technologische kunst en lichamelijke interactie.“

A randomly chosen character observes the spectator from a glass screen. From his pose and the framing of the shot, he seems to be waiting to be portrayed.

The character will carry on like this, breathing and blinking, until the spectator touches the screen. Then, he will come out of his immobility responding with the same gesture, placing his hand and gaze on the user’s hand, following any route it follows. Trapped in this small sequence of gestures that reminds of La Invención de Morel, by Adolofo Bioy Casares, the portrayed character meets the real one at an instant of simulated communion, which expresses both the will for communication and the impossibility of it actually happening.

The interactive Videobooth
After interacting with the portrayed character, some spectators will probably wish to become themselves part of the work by being portrayed. To serve this purpose, the second phase of the project will involve creating a videobooth. This videobooth will look as a large photo booth, but it will record video instead of taking pictures. When inside the booth, the user will have to perform a set of movements in a game-like environment. Frontera’s program will then turn the resulting video sequences into an interactive portrait. After completing the recording session, the user will be able to play with his image, or allow others to do so. The videobooth is to Frontera’s interactive portrait what a camera is to photography.

TECHNICAL DETAILS:

Open source alternatives
The first version of the portrait used proprietary software. For Frontera V.2, all used software was replaced by open source software.

Motion tracking
Frontera’s motion tracking application uses Intel's OpenCV library to read out the two cameras which are mounted on the portrait's frame. The cameras are continuously detecting the position of “blobs” (usually hands) in the image, and, combining these two detections, generates an (x,y) pair which it sends out over OSC, the Open Sound Control protocol, for use in external applications. The program is configured using a GUI, which lets you control all aspects of the application, from detection thresholds to optional visualization.

Recording and playback
For Frontera v.2, the videobooth uses a camera from Elphel Inc. to record video. This camera is “open hardware”, meaning that all its specs are opened to the public. The camera is a linux machine by itself, connected (and powered) over ethernet to a host computer or directly to the network. The video booth uses a combination of standard open video tools, 'mencoder' for the re-encoding of the video and 'xjadeo' for the playback engine.

Furthermore, for storage of the motion tracking data in combination with the recorded footage, a sqlite database is used. All interaction between these programs is again handled by the OSC protocol.

Team:

Art Direction: Lilia Perez Romero
Programing engineers: Robin Gareus, Arjan Scherpenisse
Physical design: Eelco Wagenaar

https://sourceforge.net/projects/frontera
http://www.liliaperez.net
http://www.on-signal.org
http://www.gareus.org
http://www.springer.com/computer/information systems/book/978-3-540-79485-1