Can an algorithm find the saddest song ever written? We can’t expect a completely objective answer, music tastes are subjective, but we should be able to reason logically about music. And wherever there is logic, we can create algorithms that help us answer a question.
Here is the type of verbal argument the logical, mathematical part of me would be willing to accept. I am going to go for Adele’s ‘Someone Like You’. Part of my argument is the scale of the effect of the song: it has made millions of people around the world cry. The other part of my argument revolves around the way she uses the phrase “never mind” to start the chorus. It is at that point Adele turns her frustration on herself, and it is there the sadness lies. The carefully controlled best wishes for the future, the second verse history of lost love told in compact form, and the closing reflections on the meaning of relationships. Emotionally complex words, sung perfectly, are set to simple piano chords. ‘Someone Like You’ is the ultimate sad song.
Spotify were quick to realise the importance of algorithmic classification of the emotional content of music. With several alternative music streaming services available, being competitive means delivering the best suggestions for new music and creating playlists that users enjoy. With 30 million songs available, it is impossible for humans analyse where each of these songs individually.
A first step in automatic classification is to exploit our listening patterns. If two songs are both listened to by lots of different users, then if you listen to one of the songs the algorithm is likely to suggest the other to you. But this is just a starting point. The most important recent innovation in classification has come from the start-up Echo Nest, which Spotify acquired three years ago.
Echo Nest have developed a method for measuring the emotional impact of music. One of the Echo Nest team, Glenn McDonald, used their algorithm to create the ‘Every Noise at Once’ project, an online interactive sound cloud of 1513 music genres. To create the cloud, Glenn and his colleagues categorised tracks on 13 different acoustic dimensions, such as energy, emotional valence, bounciness and liveness.
The process combines human analysis — -where test listeners perform pairwise comparisons to categorise songs as sadder or happier, bouncier or more sombre — -with automated measurement — -where the computer uses these human inputs to classify millions of other songs. There are challenges. For some time, the Echo Nest algorithm couldn’t distinguish between singing and banjo playing. To fix the problem, the engineers played lots of banjo and vocal tunes to the algorithm, until it ‘got’ the difference.
Glenn, who was allowed to choose his own job title when he moved at Spotify, calls himself a Data Alchemist rather than a Data Scientist. “I’m not searching for abstract truths, I’m trying to find music people will respond to”, he told me.
Spotify’s algorithm does reliably identify sad songs. They lie in an area of the soundscape with low values of both emotional valence — -measured by identifying the sounds that make people feel positive (high valence) or negative (low valence) — -and energy — -which measures the intensity, the unpredictability and the dynamic range of music. The algorithm agreed with me about ‘Someone Like You’. It has valence 29% and energy 32%, putting it almost right in the middle of the ‘sad song’ quadrant.
Not every song in this quadrant is sad — -some are bitter, melancholy or despairing — -but it is here that most of the truly emotional songs can be found. There is also a Happy quadrant, where we find ‘Jump (For My Love)’ by the Pointer Sisters, Justin Timberlake’s ‘Can’t Stop the Feeling’ and, perhaps unsurprisingly, Pharell Williams hit ‘Happy’.
For me, the sadness in a song builds just as much on the lyrics as on the sound. Data analyst, Charlie Thompson, agrees. His favourite band is Radiohead. “It is the context of the whole song which creates the emotion, the musical sadness combined with the lyrical sadness”, he told me. His favourite sad Radiohead song is Videotape, “Every time I listen to it, I find something new.”
To investigate the role of lyrics in creating emotion, Charlie extracted data from the music community Genius, that has annotated lyrics to over 25 million songs. Charlie’s algorithm then measured the proportion of sad words, such as “hate”, “fall”, “kill” and “leave”. Combining this lyrical measurement with Spotify’s emotional valence measurement, Charlie came up with an overall measure of ‘gloom’ in a song. Top of Radiohead’s gloom index was ‘True Love Waits’, from their most recent studio album.
To better understand his algorithm, I asked Charlie if he could evaluate a set of happy and sad songs using his method. The results are shown below, broken down in to musical and lyrical sadness.
‘True Love Waits’ has a certain haunting quality, but I wouldn’t agree that it is the emotionally saddest song ever. It lacks the edge of ‘Someone Like You’, the version of ‘Hurt’ recorded by Jonny Cash, or Radiohead’s own mega-sad hit ‘Creep’. Charlie’s algorithm doesn’t pick up on the sarcasm of ‘Happy Little Pill’ or the double negatives of ‘Can’t Stop the Feeling’. Algorithms still only take us so far in evaluating emotional affect, and don’t yet capture the way combinations of words create emotion.
While writing this article, I spent a couple of weeks re-listening to songs I find truly sad: ‘The Drugs Don’t Work’ by the Verve; ‘Daddy’s Gone’ by Glasvegas; ‘Back to Black’ by Amy Winehouse and ‘Suicidal Thoughts’ by ‘The Notorious B. I. G.’. A few years earlier, when I heard ‘Daddy’s Gone’ for the first time — -driving to work, late for a meeting, just having screamed at my kids in a futile attempt to get them ready for school on time — -I had to stop the car to cry. The bluntly told story of a boy without a father in his life and, ultimately, the loss experienced by men who turn their back on their families, highlighted my own ungratefulness for my privileged position and brought back to me the working class morals of the Scottish town I had left when I was 18. These are patterns and connections present only in my own head, and cannot be captured by a automatic recommendation system.
Spotify picks up on our listening patterns. Every recommendation it gives, from ‘Song Radio’ to the ‘Just For You’ playlists, uses the music classification system developed by Glenn McDonald and his colleagues. So when I now put on the ‘Discover Weekly’ service the recommended songs had, based on my recent music choices, become much more sombre. But I was frustrated with its suggestions. They didn’t have the same emotional affect as my sad favourites.
I told Glenn that I often found myself flipping by song after song, without fastening for any of the recommendations. I expected him to be disappointed, but he told me, “we can’t expect to capture how you personally attach to a song”. Spotify playlists work very well for music at parties but are less able to supply us with truly personal listening experiences.
“It’s a collective thing”, Glenn told me, “we do very well at generating playlists for social occasions and the number of skipped songs is low. But if we are trying to suggest a new song to you as an individual then we’re satisfied if you like every tenth recommendation.”
Glenn talked about the songs that were special to him, the ones that helped him understand more about relationships, about how we should recognise our common bonds despite differences in race, religion and sexual orientation. He pointed me toward ‘Home is Where the Heart Is’ by Sally Fingerett. “It is the song that has most often caused me to cry while listening to it”, he told me, “and I have cried many many times, despite knowing every time how it affected me previously”.
During the first minute listening to the song, I found the lyrics a bit corny. But as it developed the less logical part of me started to understand how Glenn related to it. I thought about how even the man tasked with algorithmically categorizing our music could be so deeply affected by a song in a way he doesn’t properly understand. And I started to cry too.
You can read more about Glenn and others like him in Outnumbered.