Neural Interfaces as Art: How Does Neurotheater Work?
The first performance of the neurotheater took place at the Geek Picnic festival in late June 2017. The new type of dramatic art is based on reading the actor’s emotions using neural interfaces and sharing them with the auditorium through light, color, music, and rhythm. This show is one of the best examples of a trend that is gaining popularity - Art & Science. The project created by a cross-disciplinary team of the Art&Science Institute at ITMO University was introduced to the visitors of the festival of modern technologies, science, and art. ITMO.NEWS has talked with an employee of ITMO University’s Higher School of Lighting Design, Darya Cheremisina (she was working on the performance) and found out the fundamental difference between the new type of theater and the traditional one and what problems the show’s creators have to solve.
The idea of the neurotheater consists of studying the bioelectrogenesis (is the generation of electricity by living organisms). The nerve impulse that is continuously created by a human brain is one example of this process. In our everyday life, this process manifests itself in physical feelings: vision, sense of touch and sense of smell. Artists learned how to record and decipher these impulses and express them through theater.
Today the rapid growth of neural interfaces has to do with new technical possibilities: the facility became accessible and people started experimenting with it in the art sphere too. However neural interfaces were intended to solve medical problems (wireless removal of the electroencephalogram).
What is the difference between the neurotheater and other theater genres that use advanced technologies?
The fundamental dissimilarity of this genre from others is that we bring the inside feeling of the actors out to the auditorium. In fact, there are no words, we take emotions right from the actor’s head and show his feelings at that exact moment. These emotions command the course of the performance, there is a cyclic reaction when an actor sees his response to the world around in a visual form and changes himself and everything around too. This is a never ending process.
That means that the same show can’t be repeated twice?
The shows never repeat because a person always experience the world in different ways - today he is not the same as he was yesterday. The overall structure of the show is similar because we rehearse and practice but the sounds and music - the reflection of what the actors feel at the moment will never be the same.
How do you cast the actors? Are you interested in some concrete character types?
The actor has to be open. He can’t be a closed person. We need someone who can easily control his feelings and know how to change them. When we rehearse we reveal the identity of each actress because there’s a good reason for everything in the performance. The colors that expressed their emotions were used because they describe these women, their taste and their inner world. During our rehearsals, we asked the actresses to recall a certain emotion, either by remembering something positive and right after something negative and and we wrote down what we observed. This correlation between negative and positive emotions was very important for us. An actor has to be able to easily cross the line between these two opposite meanings. These are the kinds of people that we need to work in the neurotheater.
How do you run the rehearsals?
During the rehearsals, we train the neural nets. The point is that it needs to be constantly trained. In fact, the neural net has to be trained for a concrete person. The module that is connected to a neural net of a certain actress can’t be used by another person to get the desired effect because this net has already been trained for it's owner. The neural net has to work with one person and his emotions constantly for one or one and a half months. If we want another character in the play we’ll have to teach the neural net for one more person.
Who made the team that was working on the show performed at Geek Picnic?
In our team there were two media artists - Yury Didevich and Sergey Blagodetelev. They were working on it and I assisted them. The actresses were girls from the dancing group STAGE DF and Aleksander Väljamäe from Tallinn University was our supervisor. Altogether we worked on the project for two months.
Which neural interfaces were used in the performance?
We used the Muse interface. This is an off the shelf product that is publicly available, however, the methods of working with it are exclusive.
Muse is able to read the length and quality of waves coming from a human's brain. Altogether there are five categories of waves: alpha, beta, gamma, theta, delta. The difference between them (between waves of left and right brain) usually doesn’t count but we worked in a reversed system. Which means that alpha-beta asymmetry gave us the result that the visitors could see on the Geek Picnic. Usually, the waves only measure (the frequency, the modulus etc.) but we worked for their correlation. We attached this dissimilarity to different conditions of actresses that were set for each girl personally.
What kind of picture appeared in front of the auditorium?
The performance took place under a dome, a square platform was located in the middle. When the visitors were entering, the actresses were sitting in meditation poses. When everybody sat down the music generated by girls’ brain started to play. They were holding spheres in their hands, and when they started to act - opened their eyes, the spheres lighted up with a certain color and actresses started to dance, interacting with each other and the object in their hands, listening to the music that appears right from their heads - it came out as a dancing performance. Also, there was a projection onto the dome. Besides the individual spheres that were reflecting one’s emotions there was also a big sphere that was showing all emotions mixed together and rhythmically producing the graphics of the actresses’ feelings. The show lasted for 10-12 minutes in total.
What is happening to the performance right now, are you developing it?
Yury Didevich is planning to bring the technologies as well as the acting to perfection together with ITMO University. He particularly likes working in a domed space because of his interest in structures and forms. In this particular case, it’s a kind of space and a synesthetic sphere.
What kinds of platforms suit such a performance? In Russia this format of show isn’t developed yet, how does the audience like it?
The sphere is big. These kind of performances can be shown only in exhibition spaces or as a part of a commercial project.
Hardly anybody in Russia develops the area of neuro and art. Yury Didevich is one of the few people in our country. We want to bring this idea to the masses and make it more suitable for any person. We want to make it such that even an unprepared person can wear the headset and learn something completely new about himself. Our final goal is to work with ordinary people.
When it comes to the reaction, the performance that we introduced at Geek Picnic was our first experience of such a show. However we got a public reaction straight away - after the show people were asking if they can also wear the fitting, how can it be used further and will these kind of things be accessible to ordinary people, not actors.
Will people get the possibility to interact with each other and the environment by using the headset or does it only suit professionals?
Yes, there will be such a possibility. I’m working on it. The idea is that anyone could enter the platform and see a certain play, a narrative that is controlled exactly by the person’s condition. The play is unconsciously built by him.
How does the theater develop with the advent of technologies? Some say that we are returning to being interactive when the boundaries between the stage and beyond it are becoming blurred.
It’s true. Moreover in neurotheater there is a completely new level of interaction. It’s possible to pull out an actor’s emotions right from his head which means that we leave any intermediaries like speaking or acting. This is a direct impact on the auditorium. Besides everybody influences the atmosphere of actors which means that they influence the whole play.
How else is the neurotheater distinguished from other genres of dramatic art? What about the decorations?
The decorations are more minimalistic, everything becomes more generative, happening in real-time including music. The latest international trend is when a theater uses new technologies and puts the actor in the spotlight. With the advent of technologies, people started forgetting that the person should be in the spotlight, that the technologies are just a support tool for our self-knowledge and self-expression.
Before performing do you tell the audience about what they’ll see or do you just let them understand the process by themselves?
This kind of genre has to be explained. Before the play starts we need to explain what it is, how it works etc. If a person just comes and looks at the show he doesn’t understand what is exactly happening because there’s no concrete action, it’s all abstract, made in the modern art style when something is happening but there’s no culmination or conclusion in it’s traditional understanding. However, we can do both culmination and conclusion in the future when it’s going to be a full-fledged 40-minute long play. In the Geek Picnic version all those phases were present but in a reduced scale. The goal was to show how it works.