Play a Kandinsky
Immerse yourself in the colours of sound with Google and “Play a Kandinsky”
The all-digital “Sounds Like Kandinsky” project produced in collaboration with Google Arts & Culture offers an immersion into the world of the pioneer of abstraction. It is also an invitation to discover the painter’s exceptional gift of synaesthesia (“seeing” music in colours) through the fun and immersive "Play a Kandinsky" tool.
Few people know that, like David Hockney and contemporary musicians Pharrell Williams and Billie Eilish, the Russian painter and art theorist Wassily Kandinsky saw music in colours, meaning that he associated each colour with a sound and vice-versa (a phenomenon experienced by only 4% of people).
“Play a Kandinsky” uses machine learning and artificial intelligence. How does it work?
Meeting with the sound artist Antoine Bertin
AB — We used a tool called Google Transformer, which is what is called a "neural network". In short, it is an algorithm that has learned what music is by itself. Google “fed” it almost everything there is, including classical, pop and jazz, and the algorithm has been trained on thousands of hours of music. It has listened to just as much Chopin, Tchaikovsky or Bon Jovi as the Beatles! We then uploaded Kandinsky's famous music archives, which included music for the piano or wind instruments such as flutes or oboes, etc. The algorithm, which is called an “unsupervised” algorithm, uses this to generate melodies by itself.
What is the role of humans in all that?
AB — It was important for us to let the machine express what music Kandinsky might have listened to and not to direct it as though it were simply a generator of propositions censored by humans. We obtained a sort of “fossil” of what Kandinsky might have heard, and from then on there was a lot of interpreting to achieve the universal. We had lots of proposals that would have been considered “clumsy” or “inelegant” by a musician. My role was to act as an intermediary between the machine and Kandinsky's theories. I was a bit like a “code DJ”, sampling and assembling different elements. The result is a sort of modernisation, an updated version of what Kandinsky may have listened to, because there are synthetic sounds that he could not have heard at the time.
What have you learnt from Kandinsky by spending time with him in this virtual way?
AB - I enjoyed getting to know him. I’m quite fond of him now, he’s a fellow artist! But what struck me most is that while his paintings were very lively and poetic, in real life he was very meticulous and scientific, almost geeky, to the extent that as I worked, I couldn’t help wondering if it was the same person! I think he needed a rigid framework to progress. Kandinsky used music to paint: it was his path to abstraction.