from 11-07-2008 until 26-07-2008

With Frontera v.2 visitors could play with their own portraits. This interactive installation consisted of two main components: a video booth and a playback screen. The video booth served as a tool with which spectators could create interactive self portraits that were shown on the playback screen and reacted on movement as if they were dissociated mirror images.

In an age where one ruling paradigm regarding technological art is physical interactivity, it’s only natural that one of the oldest visual representation practices, that of portrayal, transforms accordingly. This fact has been acknowledged in literature and cinema, thus interactive portraits have become integrated into our image of the future. Frontera v.2, by Mexican artist Lilia Perez Romero has been created within this scope of works and it could represent a first approach towards realising what others have envisioned. This interactive installation consists of two main components: a video booth and a playback screen. The video booth acts like a tool with which spectators can create interactive selfportraits. These portraits are shown on the playback screen and react to the movements of the spectator as if they were dissociated mirror images. Touch and communication are key ingredients in this project for which an open source application was developed.

Lilia Perez Romero was an artist in residence at the NIMK from April – july 2008. She continue her research at the Jan van Eyck Academy until September 2009.

The Portrait

A randomly chosen character observes the spectator from a glass screen. From his pose and the framing of the shot, he seems to be waiting to be portrayed.

The character will carry on like this, breathing and blinking, until the spectator touches the screen. Then, he will come out of his immobility responding with the same gesture, placing his hand and gaze on the user’s hand, following any route it follows. Trapped in this small sequence of gestures that reminds of La Invención de Morel, by Adolofo Bioy Casares, the portrayed character meets the real one at an instant of simulated communion, which expresses both the will for communication and the impossibility of it actually happening.

Frontera v.2 / portrait is an interactive installation that works mainly with real-time video manipulation in response to the user's actions. Technically speaking, this work is based on computer vision and motion tracking technologies. A first version of the interactive portrait (Frontera v.1) was developed at the Center for Multimedia, Mexico, and has been shown in numerous venues. During the residency at Montevideo, the portrait will be improved by developing an open source toolkit that may also be used by other artists. In Frontera v.2, a new component, the videobooth, will be added to the installation and made available for public use.

The interactive Videobooth
After interacting with the portrayed character, some spectators will probably wish to become themselves part of the work by being portrayed. To serve this purpose, the second phase of the project will involve creating a videobooth. This videobooth will look as a large photo booth, but it will record video instead of taking pictures. When inside the booth, the user will have to perform a set of movements in a game-like environment. Frontera’s program will then turn the resulting video sequences into an interactive portrait. After completing the recording session, the user will be able to play with his image, or allow others to do so. The videobooth is to Frontera’s interactive portrait what a camera is to photography.

TECHNICAL DETAILS:

Open source alternatives
The first version of the portrait used proprietary software. For Frontera V.2, all used software was replaced by open source software.

Motion tracking
Frontera’s motion tracking application uses Intel's OpenCV library to read out the two cameras which are mounted on the portrait's frame. The cameras are continuously detecting the position of “blobs” (usually hands) in the image, and, combining these two detections, generates an (x,y) pair which it sends out over OSC, the Open Sound Control protocol, for use in external applications. The program is configured using a GUI, which lets you control all aspects of the application, from detection thresholds to optional visualization.

Recording and playback
For Frontera v.2, the videobooth uses a camera from Elphel Inc. to record video. This camera is “open hardware”, meaning that all its specs are opened to the public. The camera is a linux machine by itself, connected (and powered) over ethernet to a host computer or directly to the network. The video booth uses a combination of standard open video tools, 'mencoder' for the re-encoding of the video and 'xjadeo' for the playback engine.

Furthermore, for storage of the motion tracking data in combination with the recorded footage, a sqlite database is used. All interaction between these programs is again handled by the OSC protocol.

Team:
Art Direction: Lilia Perez Romero
Programing engineers: Robin Gareus, Arjan Scherpenisse
Physical design: Eelco Wagenaar

https://sourceforge.net/projects/frontera
http://www.liliaperez.net
http://www.on-signal.org
http://www.gareus.org
http://www.springer.com/computer/information systems/book/978-3-540-79485-1