Should You Use an LLM to Analyze Your Music?

LLMs have made remarkable progress in music understanding. With audio input, they can now produce detailed descriptions of a track, which has naturally caught the attention of music catalog teams. But can you trust the output every time, for every track?

Not on its own. Recent benchmarks show that a model can produce a convincing answer even when it hasn’t accurately interpreted the recording itself. For catalog metadata, that’s a problem. Those tags become the basis for search, discovery, and decision-making, so they need to describe what’s actually in the audio.

Cyanite Senior Research Engineer Benno Weck co-authored two benchmarks for evaluating music understanding in large audio-language models (LALMs): MuChoMusic (2024) and HumMusQA (2026). We’ll look at their findings alongside Investigating Modality Contribution in Audio LLMs for Music (2026) and MusTBENCH (2026) to compare how LLMs and dedicated music analysis systems complete the same task.

What LLMs do well

LLMs are strong at describing music and answering questions about a track. We see that ability in our own testing. The best models can identify well-known songs from clean audio and return plausible tags. For someone exploring one track, that can be useful.

But generating professional catalog data is a different standard, and being good at describing music isn’t enough. A catalog team needs metadata it can trust across thousands of tracks.

The benchmark findings show how LLMs perform against that standard, and we’ll use them to compare LLM performance against Cyanite’s.

Probabilistic vs. deterministic output

An LLM generates text one token at a time, selecting each token from a probability distribution. Take the sentence “The cat sat on the…”. An LLM assigns probabilities to many possible next words. “Mat” is likely to receive the highest probability, but it isn’t the only option.

When an LLM tags a track, the answer is assembled in the same way: a sequence of probabilistic choices. And those probabilities are sensitive to the input. A small change to the recording or prompt changes the information the model receives, so it calculates a different set of probabilities. That can produce a different answer to the same question.

HumMusQA shows how this kind of input sensitivity appears in music evaluation:

• The benchmark evaluated six audio-language models on 320 expert-written music questions.
• Every question was asked four times with the same answer options presented in a different order each time.
• Gemini 2.5 Flash, one of the models evaluated, gave the same answer across all four runs for only 35% of the questions.

LLM generation can be made deterministic by setting the temperature to zero, meaning the model always picks the most likely next token. MusTBENCH evaluated every model that way. The researchers then tested whether the models could identify when musical events occurred in a song, such as when an instrument entered or when the mood peaked. The models still struggled with those tasks.

Cyanite was built for repeatable results. Our music understanding runs on an ensemble of specialized machine learning models, each trained to predict one attribute: BPM, genre, mood. Each model outputs a numerical score, and the same audio produces the same score. We turn those scores into metadata while retaining the confidence behind every tag.

Is the answer grounded in the audio?

When an LLM identifies an instrument or assigns a genre, it’s not always clear whether that answer is grounded in the recording or is a plausible guess based on context.

In HumMusQA, the researchers replaced the music with static noise or silence and asked the same 320 expert-written music questions. Gemini 2.5 Flash achieved around 38% accuracy with noise, compared with around 56% with the real recordings. So while its performance improved with the real audio, the model still answered many questions without it, showing how heavily it relied on the text.

The 2026 study Investigating Modality Contribution in Audio LLMs for Music looked specifically at how much audio and text contribute to LLM answers about music. For the more accurate of the two models tested, only about a fifth of the answers came from the audio. The rest came from the text of the question and answer choices.

Both studies share the same pattern: a meaningful share of LLMs’ answers comes from the available text, not the music itself.

In contrast, Cyanite’s models work from a single input: the audio signal. They don’t receive filenames, metadata, or text that could influence the prediction. Every tag is derived from the audio itself.

The time aspect

Catalog metadata helps narrow thousands of tracks down to a shortlist. The remaining tracks still need to fit the brief. Does the guitar only appear in the intro? When do the vocals enter? Without that information, someone still has to listen through every candidate.

That depends on a model being able to connect musical events to the correct moments in a song, which is what MusTBENCH was created to evaluate. The benchmark asks models to identify when an instrument or vocal enters or exits, order musical events correctly, and locate changes in emotional intensity.

LLMs still struggled with temporal grounding:

• They often recognized that a musical change had occurred, but struggled to place it accurately in time.
• When predicting when a sound stopped, their answers became less stable later in the track and often defaulted to round timestamps such as 120 or 180 seconds.
• Some models even predicted events beyond the end of the recording.

The paper concludes that LLMs still struggle to place musical events accurately in time.

At Cyanite, we make predictions at least once every 10 seconds throughout a song, making time-based metadata possible. For each instrument, we report whether it is absent, partially present, frequently present, or throughout. The app’s Instruments view combines those presence tags with per-segment curves, showing how each instrument appears across the full track timeline. 

What the benchmarks tell us 

These benchmarks demonstrate real progress for LLMs in music understanding, along with the limits current models still have in professional catalog work.

General-purpose LLMs can perform many different tasks, but they remain consumer products built for conversation. Cyanite is a professional tool built to support human and agentic work in the music industry. Instead of relying on a single general-purpose model, we use an ensemble of specialized machine learning models, each trained to predict a defined musical attribute.

That focus is what makes the difference for catalog work: the same recording produces the same metadata, with the confidence behind every tag available, so search and licensing decisions rest on data that holds up.

FAQs