Emotions are core to human experience. Love them or hate them, they influence the directions our individual lives and society take. In the 21st century, technology really started engaging with emotions. Ranging from artificial intelligence, robotics and biometrics to gaming, virtual reality and social media, technologies are collating, assessing and triggering our emotions. These technologies are even creating new feelings, some of which have eluded all our attempts to describe them.
Stine Deja & Marie Munk, Synthetic Seduction, 2018, Annka Kultys Gallery, photo: Damien Griffiths
In the Real Feelings exhibition, which is on show at the HeK (House of Electronic Arts Basel) until mid-November before moving to the MU Hybrid Art House in Eindhoven next March to June, we explore the rapidly changing relationship between technology and emotions through the eyes of 20 international contemporary artists.
The idea came from our curiosity about how artists are approaching the increasing interactions between technology and feelings, and by our wish for them to have a space in which to challenge, provoke, and explore how technology is representing, influencing and changing our emotions – the highs and the lows, the pleasure and the pain.
We started with this research way before the global pandemic spread. In our curatorial exchanges we spoke of how the climate crisis was one of the defining global issues in recent years, and how the prospect of mass extinction was creating increasingly collective emotional responses with new, complex feelings of grief and fear. Now with the pandemic as the most immediate global emergency, having more instantaneous and visible consequences for human-life, feelings are becoming even more a mass event. Every emotions is being triggered ranging from fear, anxiety, panic and distress to resignation, anger, relief, and in some cases hope for a better society.
From the beginning of the lockdowns, technology has become the glue that keeps us connected. It is the crucial tool that allows us to share, communicate and explore our feelings whilst maintaining a 1.5 metre distance from each other.
So the subject of emotions and technology has never before seemed more relevant. Nor has it been more fraught with new emerging possibilities of a different kind of crisis – that of the potential end of individual freedom. In fact, in this global crisis we are experiencing a new relevance of our emotions that has to do with the radical change in our life situation. Fear and uncertainty determine everyday life, more consciously than before – but differently depending on the generation. We openly fear disease and death brought by a new invisible “enemy,” not only personally but also collectively as a society, and this fear seems to lead to an urgent need for control and protection.
Countries like China, South Korea and Singapore managed the first wave of the pandemic in 2019/2020 by using mass surveillance and data collection in order to track and control their populations – technology already embedded and interwoven into the fabric of their societies. Historian and philosopher Yuval Noah Harari points out that the collecting of biometric data has an inherent danger because in the future more governments may become totalitarian and use biometrics for ends less protective than our collective health: “If corporations and governments start harvesting our biometric data en masse, they can get to know us better than we know ourselves, and they won’t just be able to predict our feelings, but also manipulate them to sell us anything they want – be it a product or a politician.”
There is also the reality that we are losing touch with our feelings and becoming less emotionally intelligent because we cannot “read” emotions in others when we are with them physically, let alone when we see them on a screen.
In the USA research shows that children are being taught how to recognize emotions in other people because their ability to do so is failing in the digital age when access to smartphones and tablets begins in babyhood. There is also increasing research into the lack of emotional development and self-regulation in young children, as anxiety and depression associated with screen time become more of a constant in their lives than interpersonal activity and exchanges. This has implications in the future as to how technology will alter social behaviour, especially in the way we are intimate or present with one another.
Real Feelings is divided into thematic areas that explore just some of the many issues that the accelerating interactions between emotions and technology raise today. In the context of this text, only a few examples from the 20 artists in the show are highlighted.
Shinseungback Kimyonghun, Mind, 2019, interactvie installation, courtesy of the artists Lauren Lee MCarthy & Kyle McDonald, Vibe Check, 2020, animation still, commissioned by HeK and MU for Real Feelings Justine Emard, Co(AI)xistence, 2017, video still, courtesy of the artist
With the Internet of Things (IoT), the objects surrounding us have become smart. They store data, know our preferences and take on simple tasks. In 2019 the number of active IoT devices reached 26.66 billion. During 2020 experts estimate the installation of 31 billion IoT devices.
Part of this increasingly networked world are the “smart assistants” such as Siri, Alexa or Ok Google, who play the desired playlist for us, dim the light or fulfil other wishes. French artist Antoine Catala's installation Everything is Okay: Season 2 (2018) alludes to this increasing smartness of our homes. The installation consists of several kinetic objects: a plastic bag, a T-shirt and socks, all of which have a smiley kind of face and are breathing, some of them even singing. They all share the title “I am here for you” and there is a wall piece that assures us: “Don’t Worry.” The objects refer to our technological interaction with many smart objects today, and at the same time they are a reference to the transformation of language itself through the increased use of emojis.
The connection between emotion and things is also the subject of How happy a Thing can be (2014) by British artist Cécile B. Evans. The video concentrates on three humble everyday manufactured personal objects – a screwdriver, a comb and a pair of scissors. The things are imbued with a life and physicality of their own thanks to the digital realm in which they appear. An implicit message can be read into the work, that with the rise of IoT, we have become so close to the objects that survey us, and they to us, that the distinctions between us are breaking down. We too have become objects or commodities, which can be bought and sold.
Coralie Vogelaar, A research on emotion recognition software, 2018, courtesy of the artist Coralie Vogelaar, Infinite Posture Dataset, 2020, commissioned by HeK and MU for Real Feelings
Emotions are often described as the essence of human experience and behaviour; they distinguish us from most beings on the planet – except (for example) primates as proven by the pioneering work of Dutch scientist Frans de Waal. But despite emotions being so important to us, in science they are a hotly contested concept, with no universal consensus as to what emotions are or where they come from. The question, “What is an emotion?” famously posited in 1884 by the American psychologist William James, is as relevant today as it ever has been, one that is in perpetual upheaval and without a unified framework for guiding it and accumulating new knowledge.
The disagreements partly centre on identifying distinctive emotions and ones that are not distinctive, as well as on the role of the body in triggering and/or receiving emotions and how that interconnection works. The American psychologist Paul Ekman thought there were only six basic emotions: happy, surprised, afraid, disgusted, angry, and sad. He later added contempt as a seventh basic emotion shown in the face.
In 2017 there was a study claiming as many as 27 different categories of emotions that are not mutually exclusive and co-exist on a gradient, thus they cannot be sharply distinguished from each other. These emotions include horror, empathetic pain, adoration and envy.
Vibe Check (2020) by American artists Lauren Lee McCarthy & Kyle McDonald makes visitors immediately aware of this fact as they enter and leave the exhibition. It consists of a series of screens on which you see people portrayed in the exhibition to whom certain emotions are attributed. While walking into the exhibit visitors encounter cameras that film them as well as people around them. These cameras analyse the emotional responses of the bystanders and attribute these to the visitor in question, thus underlining how our emotional states are influenced by many aspects. At the end of the exhibition the visitors end up at the same screens they saw in the beginning, but now they experience their own image with the emotions they triggered in others while walking around the exhibition. In this way Vibe Check raises awareness of our emotional influence on the people surrounding us, pointing out how emotions are always part of our context, interactions with others and movement through time and space.
The fact that emotions are essential to human experience and in the 21st century are deeply implicated even in our experience of biotechnology and the relations it fosters is important in American artist Heather Dewey-Hagborg’s video T3511 (2019). A bio-hacker becomes obsessed with finding the donor of the saliva she purchased online. By focusing on this subject in the context of the growth of direct-to-consumer genetic testing services, Dewey-Hagborg’s piece raises questions about how relationships, family, and day-to-day life are likely to change in an imminent future dominated by technology and post-genetic privacy.
Clément Lambelet, Happiness is the only true emotion, 2020, photo: Franz Wamhof
The environment is one of the places where the devastation of humankind’s impact on the planet is most directly felt. Some artists in the show focus on nature and the urban environment.
The Korean duo Shinseungback Kimyonghun are fascinated by nature, technology and humanity. Their interactive installation Mind (2019) comprises of drums filled with tiny metal balls, which change and create ocean sounds based on the appearance of the audience. A camera in the centre of the room is tracking the faces of visitors and facial recognition software is interpreting the emotions, which are then translated as movements of the metal balls and the sounds they create.
Britain-based artistic trio Troika’s video piece Terminal Beach (2020) shows an industrial Kuka robot covered in fur that chops down the last tree in a desolate landscape with an axe. The fur accentuates its movements and makes the violent action of the robot seem comical and alluring. This piece evokes very complex and new emotions that have no clear definition as the viewer watches the robot’s relentless and repetitive actions, which are leading to the destruction of nature. The further twist in the tale is that the robot that was animated for this piece was trained to do this by the artists. The robot is merely following orders – given by humans whose actions have contributed to the climate crisis we are now in.
The history of research into artificial intelligence goes back several decades, to the 1950s. The mid-1980s saw increased research on neural networks and their information processing mechanisms. This "Deep Learning" or "Machine Learning,” which uses neural networks powered by an innovative new deep learning method known as generative adversarial networks (GANs), has become the dominant form of AI systems in recent years, seemingly in every machine and technical tool. It’s alarming to see what kind of manipulations of reality can be manifested by AI driven “Deepfakes”, a technology that enables anyone with a computer to create realistic-looking and sounding media.
One of the latest developments is affective computing, also known as Artificial Emotional Intelligence or Emotional Intelligence. It is about collecting data by scanning our facial expressions, voices or body language to measure and deduce human emotions.
For the past few years, technology companies have fought bitterly for the right to take data maps of our faces, because facial expressions are the most highly predictive of our emotions and therefore indicative of our behaviour, too. There are 44 muscles in the human face, so understanding what triggers these muscles and how they relate to emotion can give technology companies invaluable access to behavioural data. A number of firms such as Affectiva, Emotient (now part of Apple), and Emotion Research Lab are active in this field – turning our emotions into big business as their tools aim to manipulate customer behaviour. In China, for example, payment systems using facial recognition are already possible. As early as 2015, Jack Ma, the founder of the Chinese Internet group Alibaba, presented a beta version of the "Smile to Pay" payment system, where transactions are approved through face recognition.
In her newest piece Infinite Posture Dataset (2020), Dutch artist Coralie Vogelaar focuses on the detection and interpretation of emotionally loaded body language. For this she has a dancer move on a life-size screen that is endlessly rocking back and forth – like the gadget that cheats the step counter on your smartphone. The dancers' movements are sometimes almost robotic, then again totally free-style, and show complex and conflicting emotions. Then they are taken apart and reconstructed to be read by the machine. Our primordial brain is quite capable of “reading” this body language, but what does an AI make of all this embodied knowledge? Artificial Emotional Intelligence is also increasingly used for hiring new workers.
A new commission by the Swiss artist Simone C. Niquille, Elephant Juice (2020), explores how artificial emotional intelligence is used for hiring. Set in a bathroom, the work follows a character preparing for an upcoming automated job interview. This new form of recruitment may promise much – companies like HireVue claim their software can analyse video interviews to figure out a candidate’s “employability score” and whether the interviewee is tenacious, or good at working on a team. In the US and South Korea, where AI-assisted hiring has grown increasingly popular, career consultants now even train new graduates and job seekers on how to succeed in an interview with an algorithm.
AI may be heralded as magical and potentially able to solve the world’s problems, but as artists show it is still rudimentary; much depends on how it is programmed and what information is given to it.
Troika, Last Tree, 2020, animation still, courtesy of the artists THE FRONT ROW
A whole section within the exhibition is dedicated to our progressive interactions with humanoids and other robots, which will play an increasing role in everyday life. They are entering our homes as smart assistants, are used in schools and hospitals. We see them in more empathetic roles, for example when humanoid robots like Pepper are used to keep elderly people company or when the robot seal Paro, which reacts to strokes with movements and sounds, is used in nursing homes.
Many studies have proven that emotional attachments to robots are quickly formed. Driven by AI some of them can even analyse our mood and emotions and react accordingly, although this “empathy” is purely based on pattern recognition. Of course these developments raise a lot of ethical questions.
French artist Justine Emard’s video installation Co(AI)xistence (2017) addresses questions of co-existence and co-habitation of humans and machines. In her poetic work, the robot Alter, developed by the Ishiguro Lab in Japan, reacts to the movements and verbal contact of a dancer. There is no common language yet, only tiny points of tactile encounters and forms of contact. Through touch and encounter they appear to create an emotional bond.
In their performance Cyberia (2019) by Maria Guta & Adrian Ganea the two Romanian artists are staging a pas de deux of a human dancer and her reflections as a virtual avatar. Together they are progressing through various scenarios and landscapes, ending in a moment of intimate interaction – the desire to meet.
The gap between humanity and technology is closing quickly. Creating the emotion of empathy has been identified as a key to advancing the relationships between humans and robots. But do we as humans really grasp what empathy, one of the most complex emotions we possess, truly is? Do we understand it neurologically, evolutionarily, sensorially, or psychologically?
It is clear that, as robots continue to develop emotional capacities that are framed by their machine bodies, they may also give us insight into what it is to be human and what real feelings are – and aren’t.
Feminist theorist Rosi Braidotti says that at the core of our current posthuman predicament, in which we are caught between the drive of the Fourth Industrial Revolution and the Sixth Mass Extinction – but not its sole cause – is “the unprecedented degree of technological intervention we have reached, and the intimacy we have developed with technological devices”.
The pandemic has accelerated society’s own intimacy with technology at an unprecedented rate due to increased usage by individuals and governments. In addition, there is increased development of technology being “taught” skills of intimacy and emotion – how to recognize, interact and even trigger our feelings in a space that is one of the final frontiers between the human and technology. This new techno-intimacy may lead us to become very different beings in the future – with very different feelings from those we possess now.
We are on the cusp of a revolution of our emotions in ways we cannot entirely imagine, let alone comprehend.
Marie Munk, Cable-to-Cradle, 2020, installation view at Untitled Proje"s gallery, Vienna, photo: kunstdokumentation.com
Real Feelings is an attempt to begin that imagining and comprehension as human and machine extend their growing physical intimacy to the greatest intimacy of them of all – emotional.
Real Feelings. Emotion and Technology, HeK (House of Electronic Arts Basel) until 15 November, hek.ch
MU Hybrid Art House, Eindhoven, 26 March-6 June, 2021
This text is based on the essay, Real Feelings. Emotion and Technology, Sabine Himmelsbach, Ariane Koek, Angelique Spaninks (eds.), Real Feelings. Emotion and Technology, Christoph Merian Verlag, 2020, p. 11-29