Carlos Velasco – Multisensory Experiences

Back in 2014, while I was developing my DPhil in Experimental Psychology at Pembroke, supported by Pembroke’s Annual Fund, we launched the event “Universal Synaesthesia: The integration of the senses for design”. In this event we aimed to create a space to explore and learn about the role of the senses (such as sight, hearing, and taste, but many more!) in our experiences. We also wanted to enable a space to discuss the possibility of designing art, food, consumer, and more broadly, everyday life experiences, through the nascent area of multisensory experience design.

Today, over six years later, I, together with Prof. Marianna Obrist (UCL), are about to launch our new book Multisensory experiences: Where the senses meet technology (Oxford University Press), which elaborates on some of the ideas that I first conceived during my time at Pembroke.

 

Universal Synaesthesia event, Pembroke College, 2014.

 

What is the book multisensory experiences about?

Our experiences are multisensory in nature; that is, they consist of what we see, hear, feel, taste, smell, and much more. Even our everyday experiences, such as eating a meal in Pembroke’s dining hall or having a drink with our friends at Pembroke’s Middle Common Room (MCR), involve magnificent sensory world. Just think of the last drink you had in the MCR. You might have had a glass of whisky or wine, but the experience of that drink was not limited to the drink itself. Everything from the flavour of the drink, through the glass and its physical properties, to the ambient sounds and visual composition of the MCR, all influence your experience of the drink.

In our book, Obrist and I suggest that we can consider and capitalise on the multisensory worlds that we live in, the way in which our senses work, as well as new technological developments, to design experiences. The concept of multisensory experiences has received attention from researchers in fields as diverse as experimental psychology, human-computer interaction (HCI), marketing, and the arts. Perhaps unsurprisingly, practitioners in business (e.g., Asahi), art collectives (e.g., TeamLab), airlines (e.g., Finnair) and restaurants (e.g., Kitchen Theory) have also been focusing on developing experiences based on the multisensory elements that they mix an max in a given event.

For example, in December 2018, I worked with Chef Pablo Naranjo in Mumbai India, on a multicourse dining experience called “Awaken your senses”. In this event, several dishes were created but the ingredients were not limited to the food. In one of the dishes, which we named “sonic sip”, the key ingredient was sound. Based on research suggesting that certain sound properties can enhance the perception sweetness and spiciness in food, we gave the diners a drink, twice, each time accompanied with a different soundscape. Our aim consisted of enhancing the sweet and spicy notes of the drink, by using the respective soundscapes.

Multisensory experiences are increasingly shaped and enabled through technology. For example, it is now common for many of us to use augmented reality through our smartphones (for example, when buying furniture or even when interacting with certain brands in the supermarket) and virtual reality. Whilst these technologies that are becoming more and more popular, there are multiple researchers and practitioners working on novel sensory-enabling technologies that allow experiences beyond imaginable. For example, Nimesha Ranasinghe (University of Maine) and colleagues, have been working on an interactive drinking system that, by using digitally controlled colours, tastes, and aromas, one day may facilitate sharing flavours virtually, or at least, augmenting current drinking experiences.

The increasing use of new sensory-enabling technologies is, at times, reminiscent of The Feelies in Aldous Huxley’s Brave New World. Whilst this is exciting, it also raises questions as to what the implications of, for example, having more control over sensory delivery, are, in particular, after the accelerated digitisation of human experiences that the coronavirus has brought with it. In the book, we also discuss the implications of, and responsibilities associated with those working with, multisensory experiences.

The book ends with a call for action. Obrist and I aim to empower the reader to shape, responsibly, the reader’s own and other people's experiences by considering, and capitalizing on, the multisensory worlds that we live in. We conclude the book with a quote from Henry David Thoreau: “The world is but a canvas to our imagination”.

The virtual launch of our new book will take place on August 7th, 2020. It would be great to see many members from our community attending it. For further information and registering, click here.

 

By Associate Professor Carlos Velasco (Pembroke College, 2012-2015)

BI Norwegian Business School, Oslo, Norway.