Benchmarking in the Music Industry – Knowledge Layer of the Data Pyramid

Benchmarking in the Music Industry – Knowledge Layer of the Data Pyramid

This article is a continuation of the series on the Data Pyramid which allows to turn music data into actionable insights. In this part of the series, we review the third step of the Data Pyramid – Knowledge. For details on all articles in the series, see below: 

How to Turn Music Data into Actionable Insights: This article is the first one in the series. It reviews all layers of the Data Pyramid and shows how to turn raw data into actionable insights. 

An Overview of Data in The Music Industry: This article presents all types of metadata in the industry from factual (objective) metadata to performance metadata. It is an essential guide for music professionals to understand all the various data sources available for analysis. 

Making Sense of Music Data – Data Visualizations: This article discusses the second step of the Data Pyramid – Information and how information can be presented in the form of data visualizations, making it easier to comprehend large data sets. 

Data Pyramid and the Knowledge Layer
Once you collect the data and analyze it to get information (organized, structured, and visualized data), it can then be turned into knowledge. Knowledge puts information into context. This context can be KPI’s after a significant change or performance against the competition.

At this step, attempts to look into the future and predict outcomes are made. The more specific the problem or context you’re observing is, the more precise your findings will be at this step. The Knowledge Layer produces analytics that help benchmark performance. As Liv Buli from Berklee Online University puts it, at the Knowledge layer you can tell that the artist of a certain size sells well after performing on the TV show and use this information to guide strategy for other artists of the same size. As a result, knowledge makes it possible to look at data in the industry-specific context and understand how you compare in relation to past successes and to competition. In that regard, benchmarking and setting expectations is the final outcome of the knowledge step. 

Benchmarking can take different forms within the music industry: 

Types of benchmarking 

 

  • Process benchmarking

This type of benchmarking deals with processes and aims to optimize internal and external processes in the company. You can improve the process by looking at what competitors are doing or setting one process against another. Processes relate to how things are done in the company, for example, the process of uploading songs to the catalog. 

  • Strategic benchmarking

Strategic benchmarking focuses on the business’s best practices which is often more complex than the other two types of benchmarking and includes: competitive priorities, organizational strengths, and managerial priorities. For example, an assessment of how fans responded to the brand sound in the past can help devise a long-term sound branding strategy.

  • Performance benchmarking

Performance benchmarking compares product lines, marketing, and sales figures to determine how to increase revenues. For example, as a marketing campaign for a music release develops over time, it can reveal the most vital money channels for exposure. Sales figures can indicate how artists compare to one another in terms of profitability. 

The Music Industry Benchmarks
In this part of the article, we review some of the various ways you can derive knowledge from data and how this knowledge can be used for benchmarking. This list is not final as there are many types of data in the music industry that can be analyzed and turned into knowledge.

1. Analyzing hit songs and popular artists to discover new talent

You can utilize Cyanite’s technology to analyze what’s currently working in the music industry in different markets. In particular, you can analyze popular songs and understand what makes them successful in terms of audio-based features such as genre, mood, energy level, etc. Further, you can use the Similarity Search to find tracks with a similar vibe and feel. It then helps you discover and identify new talent which may go along the same lines as current successes. Of course, that is not the whole story of making a hit but it gives you a pretty solid foundation of hitting the current zeitgeist.  

Cyanite Similarity Search interface

2. Analyzing popular playlists to predict matches

We specifically described this use case in The 3 Best Ways to Improve your Playlist Pitching with CYANITE article. You can analyze existing popular playlists such as Spotify New Music Friday or Spotify Peaceful Piano and see what songs are usually featured. This information can then help to understand the profile of the playlist which allows you to find the perfect fit and increase the chances of getting into the playlist. It also supports you in describing the songs to the editors in the way that song is accepted.  

Algorithmic performance increasingly determines whether or not strangers hear my song. More strangers hearing my songs is how my fanbase grows—and that cannot happen unless I understand how algorithms are likely to analyze and classify my song by genre, mood, and other relevant characteristics.
Brick Blair

Independent singer-songwriter, brick@brickblair.com, @brickblair

Photo at Unsplash @heidijfin

3. Analyzing marketing and sales numbers to allocate marketing budgets and support event planning

For example, record labels can analyze the performance of their artists and adjust marketing campaigns accordingly. Insight can help change the direction of the marketing campaign, choose appropriate channels, and allocate marketing budgets more effectively. In events planning, you can analyze event venues and identify the most relevant cities and venues based on past events. 

4. Analyzing the artists’ styles to identify opportunities for song-plugging

If you’re a music publisher looking to get a placement on a new release of a successful artist, analyzing their previous style and matching it to your catalog of demos would be the way to go. 

In this case, some audio qualities of the song are not important, as it will be eventually re-recorded. To analyze songs, a regular Cyanite functionality can be used including the Keyword Search by Weights where you can search your demo-catalog by the analysis results of the successful artist on weight-specific keywords to get the most relevant results. 

5. Analyzing fan engagement to identify audience segments

You can also analyze artists’ performance by looking into fan engagement on social media and music platforms. Through understanding the fan’s demographics, interests, and lives, you can create custom audiences for new artists or deepen fan engagement for the same artist based on past campaigns. This use case has been thoroughly described in the article How to Create Custom Audiences for Pre-Release Music Campaigns in Facebook, Instagram, and Google.

Photo at Unsplash @luukski

6. Analyzing trending music in advertising to find the most syncable tracks in the own catalog.

Sync licensing which includes finding the sync opportunities and pitching specific songs can benefit from data analysis and benchmarking. Trending music in brand advertising can be analyzed to reveal the brand’s sound. This sound will then be matched to specific songs in your catalog making a strong case in the pitch to the brand in terms of sound-brand-fit. If you are interested in this use case and how data can be used in sound branding and sync licensing check out the interview we did with Vincent Raciti from TRO – About AI in sound branding.

Potential Issues and Questions
– Finding reliable data, specifying the problem/context and analyzing information can be difficult. Deriving knowledge and benchmarking involves first asking a specific question or making a specific hypothesis and then getting a proper set of data to answer/verify it. If the data set is faulty, the knowledge will be wrong and potentially even harmful to your business outcome. Additionally, at this stage of the Data Pyramid, it is easy to ignore the previous steps and not explore the deeper layers of data missing out on the details.

Knowledge is about the past, not the future. At the knowledge layer, you only have information about what happened before. Usually, information about the present (though some tools provide access to real-time data) or the future is not taken into account. It is important to remember this limitation as past performance is no guarantee to future results.

Conclusion
Raw data is usually useless unless organized and interpreted. Only then does data become information. But before that, decisions on types of metadata and the methods to extract data have to be made. This process of data accumulation and filtering can be very time-consuming. 

At the stage of Information, data is structured and organized so it can be interpreted. Information requires less time to find relevant data but there is still a lot of effort involved. 

At the Knowledge level, information is put into context and can be used for benchmarking and setting expectations. This context can be historical and involve past successes or it can relate to the position of others in the market. What kind of knowledge is derived from information depends on the initial data set and on the ability to store the memories of past successes. This process of turning data into knowledge takes a whole new form when machine learning and deep learning techniques are used as they significantly speed up the process of data collection and can memorize tons of data. However, a lot of knowledge in the music industry is still derived manually by looking at the past outcomes and trying to apply them somehow in the present.  

I want to analyze my music data with Cyanite – how can I get started?

Please contact us with any questions about our Cyanite AI via mail@cyanite.ai. You can also directly book a web session with Cyanite co-founder Markus here.

If you want to get the first grip on Cyanite’s technology, you can also register for our free web app to analyze music and try similarity searches without any coding needed.

An Overview of Data in The Music Industry

An Overview of Data in The Music Industry

This article is a continuation of the series “How to Turn Music Data into Actionable Insights”. We’re diving deeper into the first layer of the Data Pyramid and explore different kinds of metadata in the industry. 

Metadata represents a form of knowledge in the music industry. Any information you have about a song is considered metadata – information about performance, sound, ownership, culture, etc. Metadata is already used in every step of the value chain; for music recommendation (algorithms), release planning, advance payments, marketing budgeting, royalty payouts, artist collaborations, and etc. 

Nonetheless, the music industry has an ambivalent relationship with musical metadata. On the one hand, the data is necessary as millions of songs are circulating in the industry every day. On the other hand, music is quite varied and individual (pop music is very different to ambient sounds, for example) so the metadata that describes music can take different forms and meanings making it a quite complex field. 

This article intends to explore all kinds of metadata used in the industry from basic descriptions of acoustic properties to company proprietary data. 

The article was created with helpful input from Music Tomorrow:

Music data is a multi-faceted thing. The real challenge for any music business looking to turn data into powerful insights is connecting the dots across all the various types of music data and aggregating it at the right level. This process starts with figuring out how the ideal dataset would look like — and a well-rounded understanding of all the various data sources available on the market is key

Dmitry Pastukhov

Analyst at Music Tomorrow

Types of Metadata

There are various classifications of the music metadata. The basic classification is Public vs Private metadata: 

  • Public metadata is easily available and visible to the public.  
  • Private metadata is kept behind closed doors due to legal and security or because of economic reasons. Maintaining competitiveness is one of these reasons why the metadata is kept private. Typically, performance-related metadata like sales numbers is private. 

Another very basic classification of metadata is Manual vs Automatic annotations: 

  • Manual metadata is entered into the system by humans. These could be annotations from the editors or users. 
  • Automatic metadata is obtained through automatic systems, for example, AI.

1. Factual (Objective)

Factual metadata is objective. This is such metadata as artist, album, year of publication, and duration of the song, etc. Factual metadata is usually assigned by the editor or administrator and describes the information that is historically true and can not be contested. 

Usually, factual metadata doesn’t describe the acoustic features of the song. 

Besides the big streaming services, platforms like Discogs are great sources to find and double-check objective metadata. 

Discogs provides a full account of factual metadata

2. Descriptive (Subjective)

Descriptive metadata (often also referred to as creative metadata or subjective metadata) provides information about the acoustic qualities and artistic nature of a song. These are such data as mood, energy, and genre, voice, and etc.  

Descriptive metadata is usually subjectively defined by the human or the machine based on the previous experience or dataset. However, BPM, Key, and Time Signature are the exception to this rule. BPM, Key, and Time Signature are objective metadata that describe the nature of the song. We still count them as the descriptive metadata.  

Major platforms like Spotify and Apple Music have strict requirements for submitted files. Having incomplete metadata can result in a file being rejected for further distribution. For music libraries, the main concern is user search experience as categorization and organization of songs in the library rely almost entirely on metadata.

Companies such as Cyanite, Musiio, Musimap, or FeedForward are able to extract descriptive metadata from the audio file.

3. Ownership/Performing Rights Metadata

Ownership Metadata defines the people or entities who own the rights to the track. These could be artists, songwriters, labels, producers, and others. These are all sides interested in a royalty split, so ownership metadata ensures everyone involved is getting paid. Allocation of royalties can be quite complicated with multiple songwriters involved, songs using samples of other songs, and etc. So ownership metadata is important.

Companies such as Blòkur, Jaxta, Exectuals, Trqk, and Verifi Media provide access to ownership metadata with the goal to manage and track changes to ownership rights of the song over time – and ensure correct payouts for the rights holders.

4. Performance – Cultural Metadata

The performance or cultural metadata is produced by the environment or culture. Usually, this implies users having an interaction with the song. This interaction is then registered in the system and analyzed for patterns, categories, and associations. Such metadata includes likes, ratings, social media plays, streaming performance, chart positions, and etc. 

The performance category can be divided into two parts: 

  • Consumption or Sales data deals with consumption and use of the item/track and usually needs to be acquired from partners. For example, Spotify shares data with distributors, distributors pass it down to labels, and so forth. 
  • Social or Audience data. Social data indicates how well music/artist does within a particular platform plus who the audience is. It can be accessed either through first-party tools or third-party tools. 

First-party tools are powerful but disconnected. They require to harmonize data from different platforms to get a full picture. They are also limited in scope, meaning that they cover only proprietary data. Third-party tools are more useful. They provide access to data across the market, incl. performance for artists you have to connect to. In this case, the data is already harmonized but the level of detail is lower. 

Another way to acquire social data is tracking solutions (movie syncs, radio, etc) that produce somewhat original data — these could be either integrated with third-party solutions (radio-tracking on Soundcharts/Chartmetric, for example) or operate as standalone tools (radio monitor, WARM, BDS tracker). It is still the consumption data but it’s accessed bypassing the entire data chain.

5. Proprietary Metadata

Proprietary data is the data that remains in the hands of the company and rarely gets disclosed. Some data used by recommender systems is proprietary data. For example, a song’s similarity score is proprietary data that can be based on performance data. This type of data also includes insights from ad campaigns, sales, merch, ticketing, and etc. 

Some of the proprietary data belongs to more than one company. Sales, merch, ticket sales — a number of parties are usually involved here.

Outlook

Today, processes in the music industry are rather one-dimensional when it comes to data. For instance, marketing budgets are often planned merely based on past performance of an artist’s recordings – so are multiples on the acquisitions of their rights. 

Let’s look at the financial sector: In order to estimate the value of a company, one has to look at company-internal factors such as inventory, assets, or human resources as well as outside factors such as past performances, political situation, or market trends. Here we look at proprietary data (company assets), semi-proprietary data (performance), and public data (market trends). The art of connecting those to make accurate predictions will be the topic of future research on the Cyanite blog.

Cyanite library provides a range of descriptive metadata

Conclusion

Musical metadata is needed to manage large music libraries. We tried to review all metadata types in the industry, but these types can intersect and produce new kinds of metadata. This metadata can then be used to derive information, build knowledge, and deliver business insights, which constitute the layers of the Data Pyramid – a framework we presented earlier that helps make data-based decisions. 

In 2021, every company should see itself as a data company. Future success is inherently dependent on how well you can connect your various data sources.

I want to integrate AI in my service as well – how can I get started?

Please contact us with any questions about our Cyanite AI via sales@cyanite.ai. You can also directly book a web session with Cyanite co-founder Markus here.

If you want to get the first grip on Cyanite’s technology, you can also register for our free web app to analyze music and try similarity searches without any coding needed.

How to Turn Music Data into Actionable Insights

How to Turn Music Data into Actionable Insights

Decisions in the music industry are increasingly made based on data. Services like Chartmetric, Spotify for Artists, or Facebook Business Manager among others are powerful tools to enable music industry teams to back up their work with data. This process of collective data generation is called datafication, pointing at the fact that almost everything nowadays can be collected, measured, and captured either manually or digitally. Data can be used by recommendation algorithms to recommend music. This data is also an opportunity for the stakeholders to make business decisions, reach new audiences in the market, and find innovative ways to do business. 

However, decision-making from data also requires a new level of analytical skills. Simple statistical analysis is not enough for data-based decision-making. The amount of data, its complexity, and challenges associated with storing and synchronizing large data require smart technologies to help manage the data and make use of it. 

This article explains the general framework for data-based decision-making and explores how Cyanite capabilities can lead to actionable business insights. It is specifically tailored to music publishers, music labels, and all types of music businesses that own musical assets. This framework can be applied in a variety of ways to areas such as catalog acquisitions, playlist pitching, and release plans.

Data Pyramid

So how are data-based decisions made? The framework for decision-making is based on the Berklee University Data Pyramid model by Liv Buli. In the next parts we outline the 4 steps of the model and add a certain Cyanite layer to it:

  1. Data Layer
  2. Information Layer
  3. Knowledge Layer 
  4. Intelligence Layer

1. Data Layer

At the very first layer, the raw data is generated and collected. This data includes user activity, information about the artist such as social media accounts and usernames, album release dates or concert dates, and creative metadata. Among others, a technology like Cyanite generates creative metadata such as genre, mood, instruments, bpm, key, vocals, and more.  

At this stage the data is collected in a raw form, so limited insights can be made out of it. This step is, however, crucial for insight generation at the next layers of the pyramid. 

It’s also important to remember a general rule of thumb: the better the input, the better the output. That is, the higher the quality of this initial data generation, the better insights you can derive from it at the top of the pyramid. Any mistakes made in the crucial data layer will be carried in the other layers biasing insights and limiting actionability.

Our philosophy at Cyanite is to have a  reduced set of data output in the first layer to help accuracy and reliability. The data is presented in readable form in the Cyanite interface. You can see what kind of data Cyanite generates in this analysis of Spotify’s New Music Friday Playlist

Cyanite Interface

2. Information Layer

At the information level, the data is structured and visualized, often in a graphical form. A first glance at data in a visual form can already bring value and enable the company to answer questions such as: “What happened with the artist, playlist, or a chart?” 

At each stage of the pyramid, an appropriate analytical method is used. The visualization and reporting methods among others at the information stage are called descriptive analytics. The main goal at this stage is to identify useful patterns and correlations that can be later used for business decisions. 

Analytical methods used at each stage of the data pyramid

(based on Sivarajah et al. classification of data analytical models, 2016)

At this level, there are two main techniques: data aggregation and data mining. Data aggregation deals with collecting and organizing large data sets. Data mining discovers patterns and trends and presents the data in a visual or another understandable format. 

This is the simplest level of data analysis where only cursory conclusions can be made. Inferences and predictions are not made on this level. 

Instruments in Detail View

One of many ways to visualize music via Cyanite’s detail view

3. Knowledge Layer

The knowledge layer is the stage where the information is converted into knowledge. Benchmarking and setting milestones are used at this stage to derive insights from data. For example, you can set expectations for the artist’s performance based on the information about how they performed in the past. Various activities such as events and show appearances can influence the artist’s performance and this information too can be converted into knowledge, indicating how different factors affect the artist’s success

This stage employs the so-called predictive analytics that answer the question: “What is likely to happen in the future?” This kind of analytics captures patterns and relationships in data. They can also deal with historical patterns, connect them to future outcomes, and capture interdependencies between variables. 

At the knowledge level, such techniques as data mining, statistical modeling, and machine learning algorithms are used. For example, machine learning methods can try to fill in the missing data with the best possible guesses based on the existing data. More information on the techniques and advantages and disadvantages of each analytical method can be found here

Here are some contexts in which the data generated by Cyanite could create knowledge:

  • Analyzing a “to-be-acquired” catalog of music rights and benchmarking it into the existing one;
  • Analyzing popular playlists to predict matches;
  • Analyzing trending music in advertising to find the most syncable tracks in the own catalog.

4. Intelligence Layer

The intelligence layer is the stage where questions such as “So What?” and “Now What?” are asked and possibly answered. This level enables the stakeholders to predict the outcomes and recommend actions with a high level of confidence. However, decision-making at this level is risky as the wrong prediction can be expensive. While this level is still very much human-operated, machines, especially AI, are moving up the pyramid levels to take over insights generation and decision making

Prescriptive analytics deal with cause-effect relationships among knowledge points. They allow businesses to determine actions and assess their impact based on feedback produced by predictive analytics at the knowledge level. This is the level where the truly actionable insights that can affect business development are born. 

Intelligence anticipates what and when something might happen. It can even attempt to understand why something might happen. At the intelligence level, each possible decision option is evaluated so that stakeholders can take advantage of future opportunities or avoid risks. Essentially, this level deals with multiple futures and evaluates the advantages of each option in terms of future opportunities and risks. For further reading, we recommend this article from Google revealing the mindset you need to develop data into business insights.  

Cyanite and Data-Based Decision

At the very base of the Data Pyramid lies raw data. The quality and accuracy of raw data is detrimental in decision making. 

Cyanite generates data about each music track such as bpm, dominant key, predominant voice gender, voice presence profile, genre, mood, energy level, emotional profile, energy dynamics, emotional dynamics, instruments, musical era, and other characteristics of the song. This data is then presented in a visual format such as graphs with dynamic and the ability to choose the custom segment.

Based on different audio parameters, the system determines the similarity between the items, and lists similar songs based on a reference track. From Cyanite analytics, it can be derived, for example, what the overall mood of the library is and different songs can be added to make the library more comprehensive. Branding decisions can also be made using Cyanite, to ensure all music employed by the brand adheres to one mood or theme

I want to integrate AI in my service as well – how can I get started?

Please contact us with any questions about our Cyanite AI via sales@cyanite.ai. You can also directly book a web session with Cyanite co-founder Markus here.

If you want to get the first grip on Cyanite’s technology, you can also register for our free web app to analyze music and try similarity searches without any coding needed.

How to Create Custom Audiences for Pre-Release Music Campaigns in Facebook, Instagram, and Google

How to Create Custom Audiences for Pre-Release Music Campaigns in Facebook, Instagram, and Google

As a music label, you know how hard it is to promote a new artist and cut through the noise. Most music advertising agencies and labels choose to do Facebook and Instagram marketing as an easy way to start. There are some important things you should know about setting up such campaigns but there are already great guides and tips on that here and here. What we want to cover in this article are the steps you can take to identify the audience for a completely new track.

An interest-based audience is a tool that allows you to select customers based on their interests. You can target people who are fans of other artists or people who browse websites similar to your artist’s webpage. Interest-based audiences feature significantly narrows your audience to the most relevant group, thus increasing your chances of reaching the right people. 

If you are a big music label you can also use custom audiences on Facebook. You might have thought that custom audiences are only applicable to music that already gained its following, but there is a workaround. All you need to do is find similar artists who fit the roster of your label and then launch a campaign based on the same audience but for the new artist. Thus you make the most out of your advertising efforts. 

With Spotify, it’s easy to find similar artists and their respective fan communities, but when the new song is not yet released or you are breaking a new artist, Spotify algorithms won’t work. So how do you identify similar artists for a track that is not released yet? You can use Cyanite’s Similarity Search to solve that problem. The Cyanite Similarity Search compares the sound of the song you want to promote with hundreds of thousands of other tracks and finds the ones that sound similar.

At this point, kudos to Maximilian Pooschke of Virgin Music Label & Artist Service  for bringing this use case of Cyanite’s similarity search to our team’s attention.

Cyanite’s Similarity Search is an intuitive tool when I’m creating custom audiences for new artists on social media. Especially for music that doesn’t easily fit into a box, the similarity search is a great entry point for campaign planning.”

Maximilian Pooschke

Virgin Music Label & Artist Service

Now, here is a step-by-step guide on how to create custom audiences using similar artists identified by Cyanite.

Step 1. Upload music to the library view and let Cyanite analyze it

Drag and drop your music to the library view. Before you do that, you need to register for free here https://app.cyanite.ai/register

Library view
Picture 1. Cyanite library view

The library view will show some data about the song such as mood, genre, energy level, emotional profile, and more. You can explore this data or move to the next step.

Step 2. Find similar songs using Similarity Search

Click on Similarity next to the analyzed song in the library to start finding similar songs from our showcase database of around 600k popular songs. Our similarity algorithms work to offer you the most relevant and precise results and focus purely on the actual sound and feel of a song. Additionally, you will see all the same analysis data available for all the songs including Moods, Energy Level, and Emotional Profile. 

Step 3. Play around with the different filters for more granular insights

Often the magic occurs when you apply different filters. Use the custom interval, play around with tempo, genre, and key, and dive deeper into different results. Then pick the artists and tracks you find most relevant from the Cyanite suggestions.

Step 4. Enrich your findings with additional data from sources like Chartmetric

To get more details on discovered similar songs and artists, you should use other data sources to further narrow down your selection and be as precise as possible. You can check out festivals, radio stations, and/or magazines to enrich your search and select more source audiences for your audience.

Custom interval
Picture 2. Cyanite Similarity Search based on custom interval

Step 5. Go and select your audiences

Off to Facebook or Instagram to create your audiences with the popular artists you have found and selected with the Similarity Search. Use interest-based targeting and enter a similar artist’s name as a keyword. Play around with keywords for maximum results. You can use artists’ names, song names, genres, or other keywords. A good comprehensive resource on how to use and manage Facebook, Instagram, and Google ads is AdEspresso

Facebook Ad Settings
Picture 3. Facebook Ads Detailed Targeting

This is just one of many ways to use Cyanite for your purposes. You can check out this article to find out more on how to use Cyanite for playlist pitching or this one to find out how to use Cyanite to find music for your videos.

Analyzing Music Using Neural Network: 4 Essential Steps

Analyzing Music Using Neural Network: 4 Essential Steps

As written in the earlier blog article, we at Cyanite focus on the analysis of music by using artificial intelligence (AI) in the form of neural networks. Neural networks in music can be utilized for many tasks like automatically detecting the genre or the mood of a song, but sometimes it can also be tricky to understand how they work exactly.

With this article, we want to shed light on how neural networks can be deployed for analyzing music. Therefore, we’ll be guiding you through the four essential steps you need to know when it comes to neural networks and AI audio analysis. To see a music neural network in action, check out one of our data stories, for example, an Analysis of German Club Sounds with Cyanite. 

The 4 steps for analyzing music with neural networks include:

1. Collecting data

2. Preprocessing audio data

3. Training the neural network

4. Testing and evaluating the network

Step 1: Collecting data

Let’s say that we want to automatically detect the genre of a song. That is, the computer should correctly predict whether a certain song is, for example, a Pop, Rock, or Metal song. This seems like a simple task for a human being, but it can be a tough one for a computer. This is where deep learning in the form of neural networks come in handy.

In general, a neural network is an attempt to mimic how the human brain functions. But before the neural network is able to predict the genre of a song, it first needs to learn what a genre is.

Simply put: what makes a Pop song a Pop song? What is the difference between a Pop song and a Metal song? And so on. To accomplish this, the network needs to “see” loads of examples of Pop, Rock or Metal, etc. songs, which is why we need a lot of correctly labeled data.

Labeled data means that the actual audio file is annotated with additional information like genre, tempo, mood, etc. In our case, we would be interested in the genre label only.

Although there are many open sources for this additional information like Spotify and LastFM, collecting the right data can sometimes be challenging, especially when it comes to labels like the mood of a song. In these cases, it can be a good but also perhaps costly approach to conduct surveys where people are asked “how they feel” when they are listening to a specific song.

Overall, it is crucial to obtain meaningful data since the prediction of our neural network can only be as good as the initial data it learned from (and this is also why data is so valuable these days). To see all the different types metadata used in the music industry, see the article an Overview of Data in the Music Industry.

Moreover, it is also important that the collected data is equally distributed, which means that we want approximately the same amount of, for example, Pop, Rock, and Metal songs in our music dataset.

After collecting a very well labeled and equally distributed dataset, we can proceed with step 2: pre-processing the audio data.

A screenshot from a data collection music database

Step 2: Pre-processing audio data

There are many ways how we can deal with audio data in the scope of music neural networks, but one of the most commonly used approaches is to turn the audio data into “images”, so-called spectrograms. This might sound strange and counterintuitive at first, but it will make sense in a bit.

First of all, a spectrogram is the visual representation of the audio data, more precisely: it shows how the spectrum of frequencies that the audio data contains varies with time. Obtaining the spectrogram of a song is usually the most computationally intensive step, but it will be worth the effort. Spectrograms are essentially data visualizations – you can read about different types of music data visualizations here.

Since great successes were achieved in the fields of computer vision over the last decade using AI and machine learning (face recognition is just one of the many notable examples), it seems natural to take advantage of the accomplishments in computer vision and apply them to our case of AI audio analysis.

That’s why we want to turn our audio data into images. By utilizing computer vision methods, our neural network can “look” at the spectrograms and try to identify patterns there.

Spectrograms from left to right: Christina Aguilera, Fleetwood Mac, Pantera

Step 3: Training the neural network

Now that we have converted the songs in our database into spectrograms, it is time for our neural network to actually learn how to tell different genres apart.

Speaking of learning: the process of learning is also called training. In our example, the neural network in music will be trained to perform the specific task of predicting the genre of a song.

To do so, we need to split our dataset into two subsets: a training dataset and a test dataset. This means that the network will be only trained on the training dataset. This separation is crucial for the evaluation of the network’s performance later on, but more on that in step 4.

So far, we haven’t talked about how our music neural network will actually look like. There are many different forms of neural network architectures available, but for a computer vision task like trying to identify patterns in spectrograms, so-called convolutional neural networks (CNN) are most commonly applied.

Now, we will feed a song in form of a labeled spectrogram into the network, and the network will return a prediction for the genre of this particular song.

At first, our network will be rather bad at predicting the correct genre of a song. For instance, when we feed a Pop song into the network, the network’s prediction might be a metal song. But since we know the correct genre due to the label, we can tell the network how it needs to improve.

We will repeat this process over and over again (this is why we needed so much data in the first place) until the network will perform well on the given task. This process is called supervised learning because there’s a clear goal for the network that it needs to learn.

During the training process, the network will learn which parts of the spectrograms are characteristic of each genre we want to predict.

Example architecture of how a CNN can look like

Step 4: Testing and evaluating the network

In the last step, we need to evaluate how good the network will perform on real-world data. This is why we split our dataset into a training dataset and a test dataset before training the network.

To get a reasonable evaluation, the network needs to perform the genre classification task on data it never has seen before, which in this case will be our test dataset. This is truly an exciting moment because now we get an idea of how good (or maybe bad) our network actually performs.

Regarding our example of genre classification, recent research has shown that the accuracy of a CNN architecture (82%) can surpass human accuracy (70%), which is quite impressive. Depending on the specific task, accuracy can be even higher.

But you need to keep in mind: the more subjective the audio analysis scope is (like genre or mood detection), the lower the accuracy will be.

On the plus side: everything we can differentiate with our human ears in music, a machine might distinguish as well. It’s just a matter of the quality of the initial data.

Conclusion

Artificial intelligence, deep learning, and especially neural network architectures can be a great tool to analyze music in any form. Since there are tens of thousands of new songs released every month and music libraries are growing bigger and bigger, music neural networks can be used for automatically labeling songs in your personal music library and finding similar sounding songs. You can see how the library integration is done in detail in the case study on the BPM Supreme music library and this engaging interview video with MySphera. 

Cyanite is designed for these tasks, and you can try it for free by clicking the link below.

I want to integrate AI in my service as well – how can I get started?

Please contact us with any questions about our Cyanite AI via mail@cyanite.ai. You can also directly book a web session with Cyanite co-founder Markus here.

If you want to get the first grip on Cyanite’s technology, you can also register for our free web app to analyze music and try similarity searches without any coding needed.

4 Ways How We Use Music To Regulate Our Emotions In Everyday Life

4 Ways How We Use Music To Regulate Our Emotions In Everyday Life

Music listening is an integral and oftentimes purposeful activity in our daily lives. We listen to particular tracks in order to change our current emotional state or in order to maintain it. How we react to a particular song not only depends on the musical attributes of that song but on various situational and personal factors.

Originally posted on our Groovecat blog, written by Sami Behbehani , 15. March 2019

Possibilities for musical self-regulation are limitless in today’s modern society. Technical advancements such as smartphones, high-quality earphones, and music streaming have enabled listeners to access massive song-libraries from anywhere, at any time.

Consequently, individuals can immediately react to new circumstances by adapting their listening strategy accordingly.

However, the process of self-regulation through music is highly subjective and dependent on various factors.

Perceiving an emotion doesn’t mean that you feel it the same as other people

 

A song is a construct, whose single elements merge and ultimately communicate a particular feeling or atmosphere. Most likely, a listener will perceive this feeling accurately. Yet, the feeling having any effect on the emotional state of the listener is not given.

“Whether a certain song evokes an emotion or not depends firstly on the listener’s musical preference, secondly the previous listening experience and thirdly, empathy with the recording artist” – Sami Behbehani

As in movies, a certain degree of identification with the protagonist is preconditioned for the story to touch the audience. In a musical context, empathy is the precondition for a song’s story to strike interest and cause emotional contagion. Studies have shown that with an increasing degree of empathy towards a song/artist, a higher correspondence between perceived and felt emotion during music listening can be experienced.

Your listening environment influences your music selection more than personal attributes

Some recent scientific studies have shown situational circumstances to have a stronger influence on the process of music selection than personal attributes of the listener. However, capturing the essence of a situation is a complex and scientifically still relatively unexplored issue. Situations do not only include physical elements such as location, persons, weather, time of day etc. But there is also the aspect of how a person reacts towards these respective elements. This aspect even includes potential highly complex interactions between person and situation.

In our daily lives, we experience various situations that affect us in different ways and to which we react accordingly. While some of these situations occur spontaneously, others allow us to plug in our earphones or switch on our speakers. For instance: On our way to work we might get bored and hence need something to lift us up; while getting ready in the morning we might want to start the day off on a positive or energetic note; when we socialize with others we like to create a comforting atmosphere; and in order to prepare for a stressful situation we want to reach a higher state of excitement, in order to handle the situation better

Common strategies of emotional regulation

  1. Aesthetic enjoyment

Studies have shown that personal well-being is a key motive for music listening. When listening to preferred songs it makes the listener draw enjoyment from the overall listening experience. Liked music was shown to trigger the release of neurological messengers such as dopamine and serotonin, signaling pleasure and reward to the system, resulting in increased comfort. This can be interpreted as a mood-improvement process through aesthetic stimulation, which however does not modify the listener’s emotion in a specific fashion.

 

  1. Sustaining cheerfulness

Further in line with the principle of emotional regulation is a deliberate choice of songs that communicate emotions parallel with those felt by the listener. Persons experiencing cheerfulness tend to listen to happy music more frequently because they like to maintain the emotional state they are in. This is a common strategy in situations where social interaction between persons is desirable, as at parties or relaxed evenings with friends.

 

  1. Emotional Self-therapy

Another strategy that directly influences a music listener’s emotional state is utilized when experiencing negative emotion. Sad music, for instance, is highly popular amongst listeners of different genres on the one hand; and on the other hand, it can exert a strong effect on the listener. As compared to happy music which rather maintains or enforces an existing emotional state, sad or depressing songs are more commonly used for musical self-therapy. If previously mentioned mechanisms such as empathy with the song/artist, preference for the style etc. are given, sad music can mirror the listener’s feelings and therefore help to process experienced sadness, ultimately resulting in uplift.

 

  1. Stimulation

Aggressive music is a special case in itself because it can be positively stimulating on the one hand yet also expresses a negative emotional connotation on the other hand. Listening to aggressive music while experiencing feelings of aggression can have a channeling effect. Beyond that, intense music, aggressive music, in particular, enables the listener to achieve a higher degree of stimulation. This effect is consciously or subconsciously utilized by music listeners in order to: get pumped up for physical activities such as sports or dancing; motivate themselves to pull through monotonous tasks such as housework and cooking; or prepare themselves mentally for events known to include conflict and negative stress.

You are currently viewing a placeholder content from Default. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.

More Information

You are currently viewing a placeholder content from Default. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.

More Information

You are currently viewing a placeholder content from Default. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.

More Information

Implications for the future

It can be suggested that any form of maintaining or improving one’s emotional state through music falls under the category of musical-self therapy.

There is however no auditive all-around solution for daily needs since individuals vary in their personal attributes and situations exert different effects on different people. Since music recommendation algorithms rarely or not at all focus on mentioned aspects, it is unlikely for them to serve as an adequate daily regulation-tool for listeners.

Research is still at a point where new discoveries can potentially shake up the field and though there are several studies with valid findings, most likely no study will ever be able to include all parameters that fully explain human music listening behavior.

From the consumer’s perspective, the last few years of technological development have facilitated a free and goal-driven use of music. This positive development could continue in the future with tech-companies and start-ups working on new ways for music to fulfill the listeners’ potential needs.