28-01-2008

where architecture and interactive art encounter each other

by Annet Dekker

Interactive art has been around for quite some time now. Presently the designation for it appears to be shifting, however, to terms such as 'emotional architecture', 'ambient experience design', 'reactive environment' and 'relational architecture'. This is not surprising, because a lot of interactive artworks employ invisible technology which uses the body as the interface that activates the work. The manner in which we are dealing with technology in our daily lives also reflects this shift. Our relation with our environment is becoming increasingly immaterial, from mobile apparatus such as the telephone, game controllers and gps navigation systems to electronic tags on products. We move through a world that detects us, follows us, and sometimes reacts to us without our being aware of it. The technology is the point of departure, and our body is its vehicle.

These shifts have an important effect on our experience of location, space and geographic determination on both the personal and global level, but also in a digital and physical sense. Although this is not a new development – for a long time now we have made use of electronically activated sliding doors and elevators and escalators that move us along, to the point where physical characteristics are ascribed to these technologies (Gibson 1979, among others) – we have now arrived at the stage where the technology has not only become smaller, but also less visible. Without a clear interface, which in addition to being reactive increasingly often develops its own implicit reactions, these experiences become ever more complex. This makes it more important than ever that such processes be investigated critically, particularly when we are part of such environments emotionally as well as physically. This paper is part of the opening stages of an investigation into the following question: to what degree do examples from contemporary art practice in which 'emotional engagement' by means of interactivity is emerging help us to reflect on the role of omnipresent technologies and the manner in which we use them?

Interactive installations have always forced us to think about the complex experience of interaction and immersion. Much has been written about interactive installations, but in all of it little attention has been paid to the consequences that this technology has for our experience of space, and our experience of our body. The use of the term 'interactivity' still mostly reflects this tendency. Originally the term 'interactive' was used as an equivalent for circular 'simultaneous actions'. Here interaction indicates an output (in reaction to an input), in which interaction particularly denotes the effect on the manner by which the output is determined. That is, interaction is always a matter of mutual influences.

Presently, however, almost any installation that responds to an impulse from outside itself is labelled interactive. Interactivity is used for everything and its brother, from software to lamps and mobile telephones. On the one hand it is asserted that all art is interactive (because active looking creates a personal experience) and, on the other, labelling works that are made with computers as interactive is meaningless because that is the most basic function of computers. A number of artists have specifically designated their work with the term 'reactive', because their work does not just deal with the action, put particularly with its poetic qualities. In what follows, I will be focusing on installations by Daan Roosegaarde, Sonia Cillari, Rafael Lozano-Hemmer and Usman Haque. They investigate the meaning and wish for 'emotional engagement' and interactivity in which our body is employed as a natural interface. In their works they seek to extend our dynamic and emotional range by investigating conscious and unconscious relations between the participants and their relation to their environment. By using buildings, spaces and bodies as entities that are in motion, they interrogate traditional views about our bodies and the space in which we live and move. Although this is being done from different backgrounds and perspectives, it is always with the same aim, of searching out and interrogating the relation between body, space and architecture.

Remarkably enough, the works I will be discussing all come from artists with a background in architecture. From this it seems that the world of architecture is busy shifting over into that of media art, in a process in which attention focuses particularly on omnipresent technologies and the relation between people, space and interactivity. This has been a reason for me to look into the discussions going on in architectural theory. There it appears that especially the discourse around topological theory and the concept of 'flow' are of interest for obtaining better insight into the concepts and ideas of these installations. In addition, I hope that these theories will provide insight into the effects and experience of media art installations, particularly when it comes to installations that are located or presented in public space.

Daan Roosegaarde has a background in sculpture and architecture. This combination of disciplines forms the basis for his interest in the behavior of people in a space that is designed with the aid of various media and technology. His fascination with technology arose because he wanted to have the dynamic of the production process return in the end result. In his static installations he missed the direct relation between the work and the audience, and he wanted to translate the mental effect into a physical experience. By using various technologies and software, with his installations he was able to construct physical relationships between the visitors and the external features of the space. He uses the term 'liquid constructions' to describe his work.

His installation Duin 4.0 is an investigation into a new form of nature in 21st century urban space. The aim of the project is to perform research on dynamic surfaces that respond physically to the sound and movements of the user. The project is a fusion of art and technology and a merger of software, electronics and mechanics. The physical relation between the behavior of the visitor and the exterior surface of the work, as an extension of the skin, is central here. The installation is composed of a large number of light helmets that light up as soon as someone is detected; each action is followed by a programmed reaction. As Roosegaarde says, what is of primary importance for him is the poetic experience, and he wishes to offer an image of nature in the 21st century. In addition, he tries to create an idea of a liquid space. This latter emerges still more strongly in his earlier work, Liquid 2.0. Here Roosegaarde tries to take the architecture and space in which we live and adapt it to our behavior, for instance by making the space smaller if the visitor is quiet, and enlarging it when a lot of noise is made. The expansion of the installation is intensified by sounds and lamps that go on and off.

In addition, every so often, at unexpected moments, the installation displays traits of its own too. With these Roosegaarde tries to give the installation human qualities and character, or in any case to simulate them. By adding this extra layer he seeks to break through the visitor's preconceptions and expectations, even as he does with his interpretation of 21st century nature in Duin. But Roosegaarde is primarily interested in new ways of relating to architecture and the use of space, in which space reacts to the user like a second skin, creating a dynamic situation in which the space and its user become one. What at first comes across as a game with moving components, on further consideration is an impetus for rethinking the static space in which we find ourselves, and the role that we can play in this. Designed environments of this sort promote the personalisation of architecture.

In her work, the architect and media artist Sonia Cillari also makes a clear distinction between interactive and reactive. In her installation Se Mi Sei Vecino (If you are close to me) she is seeking ways to make communication visible. All her work has an emotional and physical focus, and unlike Roosegaarde she is directly concerned with the body as interface. Cillari's interest is in the participants and the moment at which they realize that the boundaries of their body extend beyond their skin. Her intention is to visualize that moment. The idea of what skin-consciousness is, how proximity, presence and touch influence our manner of seeing ourselves and how we relate to others, and how this process can be given new meaning: these are what Cillari particularly seeks to have emerge from her work. The visual representation and the audio are means of attracting people to take part, but the moment of physical contact, when the interaction is created through our senses, is what is important about the interface.

The project focuses on the interrelation between Body and Body and Body and Space. These are interactions of movement, and not of presence. By expanding someone's emotional sphere and seeking to acknowledge behavior, the work emphasizes the conscious and unconscious relation between participants, and their relation with space. This is created by mapping out electromagnetic fields; actions, movements, attraction and repulsion between participants and the human antennae are picked up directly and visualized in real time. By their movements, and especially by physical touch, the persons on the platform set sounds and images in operation; thus one can speak of both a sensory and a kinesthetic experience.

Cillari employs digital and electromagnetic energy which, using the body as an antenna, generates a dynamic form. This enables her to not only visualise communication, but she also points toward ways of investigating space in a sensory manner. Cillari shows how we, as spectators, represent the internal and external world by means of our sensory system – and the manner in which our sensors arrive at the process of consciousness in order to identify the world. Se Mi Sei Vecino makes the 'vitality affect' visible in various ways, as Brian Massumi describes an artwork as 'an explicit experience of what otherwise slips behind the flow of action and is only implicitly felt'.

More than the two artists already discussed, the Mexican-Canadian Rafael Lozano-Hemmer focuses expressly on public space, and specifically on the space surrounding historic architecture. Since 1994 Lozano-Hemmer has been using the term 'relational architecture' to denote his work. With it he wants to point to the relation between the visitor/performer and the architecture.
In his projects this relation is specifically focused on the belief that architecture structures and controls human conduct. He seeks to undermine this traditional understanding with his large installations. He calls existing beliefs about urban space into question by providing buildings with a different context with the aid of audio-visual tools.

In his first installation, Displaced Emperors (1997), he visualizes a forgotten historical link with the existing architecture of the Hapsburg Castle in Linz. Wireless 3D sensors calculated where a visitor was in respect to the building, and projected a large image of a hand at that spot. By moving along the façade with the hand, an interior became visible behind the wall – only this was not the interior of the building in Linz, but of Chapultepec castle, the principle residence of Ferdinand Maximilian, Archduke of Austria, during the Hapsburg rule of Mexico (1864-1867). In addition, for 10 schillings one could interrupt the interaction, so that the headdress of Montezuma was seen – an object which, despite passionate attempts on the part of Mexico is still in the possession of the ethnological Museum in Vienna. Finally, the visitor who was using the tracker was followed through a spot which contained the symbol of cultural property (an image created to provide buildings and monuments with cultural symbols). This installation changed the building by literally and metaphorically placing elements from the outside world on it.

The works Body Moves (2001) and Underscan (2006) are much more personal. Here he challenges the passive use of large video screens in public space. His first priority in these projects is the creation of social relations. Thousands of photographs of pedestrians on the street are projected onto the façades of large buildings (among them the Pathé cinema in Rotterdam), but the portraits are only visible when someone who is walking past throws a shadow over the image. Then the buildings and façades are transformed into playful projection surfaces on which passers-by open themselves up for others. The shadow of a body becomes the medium of conveyance. According to Lozano-Hemmer, the shadow functions as a disembodied extension of the corporeal person, connected to the body but not of it. In Underscan the relation between the projected image and the person walking past is more intense. Here too the shadow of the passer-by triggers the images becoming visible, but in this case the visitors are scanned and their route predicted by the computer. The projections then respond to the shadows by watching the passer-by, turning away, or mimicking him or her. Here the shadow is the host for the other unknown body. After seven minutes the whole field changes into a grid on which the surveillance cameras are to be seen and the positions of the visitors can be followed. Lozano-Hemmer's projects demonstrate that it is possible for social activities to give meaning to space, and that space is active in this process and not merely a clunky receptacle for social activities. The projects detect and follow the visitors, but at the same time reveal their own technical apparatus. This stimulates reflection on these systems on the part of the users, and this has the potential to transcend the specific content of the work. Lozano-Hemmer invites the visitor to think about alternative bodies, architecture and the urban experience, and then to construct them.

Looking at these works, it appears to make the most sense to not speak about architectonic space, but about a space of relation; this would widen the topological theory that is much cited in architecture. As it happens, the product is not only defined by its essential form, as it is in traditional topological theory in which the continuous development of this form, without ruptures or clear transitions, is central. The growing attention being given to relational structures which generate space that can continually be changed by our bodies in a system of interaction, can comprehend the topological tendency better, and be translated as a space of relation. Yet in these projects there is surprisingly little attention for interaction; they deal only with response systems.

Usman Haque, an architect who organizes interactive installations, digital interface designs and massive public performances, investigates this dynamic in various ways. Together with Rob Davis he developed the Evolving Sonic Environment (ESE) installation. Rob Davis is a systems designer, working in the Department of Psychology at Goldsmiths College in London. He is chiefly interested in systems that are dependent on their environment and the people who are in that space. ESE is comprised of several units of speakers, microphones and electronic printed circuit boards that hang on the ceiling. Each unit emits a sound with a different frequency. Mutual communication among the units balances the sounds and maintains a sensitive sonic ecosystem that is constructed in the space.

The communication among the various units takes place through changes in sound rather than electronically. By simply changing the frequency range, at the beginning (input) or the end (output), they communicate via a different channel. As a result, various groups begin to communicate within multiple frequency ranges; a similar linking system is also to be found in simulated nervous systems. As visitors enter the space they influence the acoustic environment and the generation suddenly changes into a new audio ecosystem. In contrast to traditional approaches to imitating neural processes, which generally involve digital image screen simulations, ESE is based on analogue components and is a physical installation in which visitors themselves can see and hear how their movements and sounds influence the nervous system. This time too the immediate presence of the public determines the acoustic environment.

ESE behaves as a 'human sensor', although there are no direct sensors built into the individual units. Haque and Davis are looking at to what extent an interactive space can be constructed, and in what way the space constructs its own representation with the aid of a network of autonomous objects. Open Burble (2006-2007) is one of Haque's latest investigations into interactive networks and structures. The installation, which is presented in public space, consists of a series of balloons connected to one another, which are collectively kept in the air by the public. Every balloon has its own LED lamp, and is linked to the other balloons by modular systems that respond to one another. The actions of the public determine the pattern and movement of the whole. Based on the same principles as ESE, Open Burble reproduces the essence of interactive systems.

In addition to being interactive works, more than the other installations Haque's projects are clear examples of how 'flow' comes into being and operates. This term comes from Luc Steels, professor of computer science at the Free University of Brussels, who introduced it in his research into the improvement of educational teaching models. Steels bases his concept of flow on the Autotelic principle, which was advanced in 1978 by the humanistic psychologist Mihaly Csikszentmihalyi. According to Steels, Flow is not a fixed state, but only comes into being when a state of equilibrium between high challenge and sufficient knowledge is reached. Should the challenge become too great for the knowledge on hand, or if the possibility for action is so overwhelmed that there is no longer any clear way to be found, anxiety takes over, particularly if there is not hope of supplementing the knowledge. That can paralyze a person, cause them to withdraw, and even become depressed. If the challenge is too small, boredom takes over, which in the long run will have equally negative consequences. The optimal level is somewhere in between, when there is a balance between challenge and knowledge. Nevertheless, this state of equilibrium will never last long. As soon as an activity has been repeated often enough, knowledge builds up to where boredom takes over, and new challenges must be sought. Thus someone who seeks an experience of flow is always seeking and moving on. Such conduct, and this sort of teaching method, is also visible in the installation by Haque and Davis. In addition to providing pleasure or esthetic enjoyment, an experience and a space of relation, it reveals to us why people are enthusiastic about interaction, and even what characteristics a system has to have to enter into a state of flow. Haque believes that future progress will lead to designed environments no longer having to fall under the category of 'interface', but can evolve into spatial 'operating systems' based on flow. ESE and Open Burble are clear examples that this is in face the case.

Artists like Daan Roosegaarde, Sonia Cillari, Rafael Lozano-Hemmer and Rob David and Usman Haque are investigating the meaning of and desire for 'relational architecture' and 'interactivity', in which the human body is taken as the point of departure. By investigating conscious and unconscious relations between participants and their environment, they enlarge our emotional range. By using space and bodies as entities which are in a state of change, they open up traditional views about space and the body to discussion. They show us all manner of ways of dealing with this newly formed environment, and help us to realize that we can and should move in flow. Should such projects be coupled with architecture, new flows will come into being that will influence architecture in both a social and political sense. In looking forward to this, I am not opting for the many moving or flickering façades, or massive video screens that are presently surfacing everywhere in public space. The majority of such reactive projects do not stimulate reflection, nor do they pose critical questions about communication or relations between architecture and its users. The most important thing about installations in public space is that they devise ways in which people themselves feel more engaged, and ultimately more responsible for the space in which we move.

Lecture presented at the NGE symposium 'Art and Public Access', University of Utrecht, October 5 and 6, 2007