How Much AI Is in This Track? Quantifying the Proportion of AI-Generated Stems in Hybrid Music Mixtures

The paper reformulates AI music detection from a binary classification task into a regression problem on a continuous AI energy ratio, α ∈ [0, 1]. The authors motivate this by noting that AI-generated music is increasingly used at the stem level, with producers integrating synthetic drums, basslines, or vocals alongside human-performed instruments, whereas current AI music detection systems are binary, treating tracks as either fully AI or fully human.

The authors define the binary detection problem as predicting y ∈ 0, 1 for a given audio signal, where y = 0 indicates human-produced music and y = 1 indicates AI-generated music. They explain that detectors exploit architectural artifacts: the characteristic spectral peaks introduced by transposed convolution layers in neural audio codec decoders, with peaks appearing at f artifact = fs/(2 × stride).

For the mixed-content setting, the track is a linear mixture of n stems, and the AI energy ratio is defined as:

α = (Σ si∈A RMS(si)2) / (Σ i=1 n RMS(si)2)

where α is the fraction of total signal energy contributed by AI-generated stems.

The reformulated task is regression: Given a mixed audio signal xmix, the task is to predict α̂ = fθ(xmix) ∈ [0, 1], where α̂ is the estimated AI energy ratio and fθ is a learned function parameterized by θ.

The authors propose a methodology for constructing controlled hybrid mixtures with three guiding principles: "(1) stem-level granularity, reflecting the modular structure of modern production; (2) artifact isolation through reconstruction, which reduces content confounds between AI and human conditions; and (3) combinatorial completeness, which provides dense coverage of α over the full [0, 1] range."

The AI reconstruction pipeline builds AI stems by encoding and decoding existing human-performed stems through a neural audio codec, instantiating the pipeline with EnCodec at 3 kbps. The AI stems in our study are codec reconstructions of real recordings, not generator output, so what we measure is the proportion of codec-generated energy under controlled conditions.

Combinatorial mix generation: Given a track with n stems indexed by I = 0, 1,..., n − 1, we define a mix specification by the subset S ⊆ I of stems replaced with their AI-reconstructed versions, producing all 2n configurations per track. Sets U0 (all real) and Un (all AI) correspond to the binary detection setting; the intermediate sets U1,..., Un−1 contain the mixed-content cases.

The methodology is instantiated on MoisesDB, a professionally mixed multi-track dataset, using 240 tracks, with stem counts ranging from 2 to 10, yielding 21,212 unique mixes across the 240 tracks. The authors release a reproducible Python library called ARIA (Authentic vs. Reconstructed Isolated Audio).

The authors compare two models sharing the same CNN backbone: a binary detector trained with binary cross-entropy on the original 0, 1 labels following Afchar et al., and a regression model with a single-unit, sigmoid-activated regression head trained with mean squared error against the AI energy ratio α. The split is 80/10/10, leaving 24 held-out tracks and 1,792 test mixtures. The binary detector achieved 99.97% accuracy and 99.98% F1 on a held-out FMA evaluation set.

Detection response as a function of α: The binary detector's output increases with α, but the increase is not linear: its median stays near 0.05 for α 0.9, showing that codec artifacts in hybrid mixtures carry proportional information about AI content, which the binary classifier picks up incidentally as a noisy and miscalibrated estimator of α. The regression model follows the diagonal across the full range.

Instrument-level sensitivity: Using tracks with stems vocals, drums, bass, guitar, the authors found that for the binary detector, drums and guitar separate most clearly, with the AI distribution concentrated near 1 and the real distribution near 0; whereas the distributions for vocals and bass substantially overlap. Notably, vocals and bass become partially separable under regression, indicating that the regressor can extract proportional information from a weaker fakeprint signal even when the binary detector cannot.

Fakeprint analysis: "Bass shows essentially no separation between real and AI conditions, whereas vocals show very subtle differences. In contrast, drums show clear separation from ∼ 2500 Hz onward... Guitar shows a distinctive crossover: the AI curve sits below the real curve in the low-mid range (∼ 500–1500 Hz) and substantially above it from ∼ 2500 Hz onward. These patterns align with the binary detector's instrument sensitivity... bass and vocal evade detection because the codec leaves a lower detectable artifact signature in their content, while drums and guitar carry strong signatures."

Regression performance: At the matched 5-second window, the regression model achieves MAE = 0.076, RMSE = 0.102, R2 = 0.85, and Pearson r = 0.93. The binary detector used as an implicit α estimator reaches MAE = 0.289 and R2 = −0.85, meaning "the estimate is less accurate than always predicting the dataset mean, even though the same detector classifies the pure endpoints with > 99% accuracy."

Window length effects: The regression model improves with longer windows (MAE 0.076 vs. 0.104, R2 0.85 vs. 0.78)... The binary detector, which is never trained on α, gets worse (R2 −0.19 vs. −0.85).

Boundary compression: The regression model exhibits mild compression near the boundaries: predictions floor at ∼ 0.05–0.10 for fully real mixtures and ceiling at ∼ 0.85–0.95 for fully AI mixtures, with the largest errors occur in the mid range, where mixtures admit the most varied stem compositions.

The authors note limitations: Our empirical findings are based on a single neural audio codec (EnCodec) at a single bitrate (3 kbps), and our mixtures are also constructed without the post-processing typical of professional production workflows (EQ, compression, level balancing, creative editing of AI stems).

The paper concludes that codec artifacts in hybrid mixtures carry proportional information about α, and the proportional-detection formulation offers a more informative target than binary classification for the realistic case of hybrid music production. The authors suggest as next steps instrument-aware detection: applying detection to separated stems before aggregation, or conditioning on stem identity during training, as well as using per-window scores to give a route to temporal localization.

Improvements to AI systems based on this paper:

• Replace binary AI-music classifiers with regression heads that output a continuous AI-energy ratio (α ∈ [0, 1]) instead of a hard AI, human label. This gives calibrated, proportional estimates of how much AI-generated content is present in a mixed track, rather than a single forced choice.

• Train on stem-level hybrid mixtures rather than only fully-AI/fully-human tracks. Build training data by reconstructing individual stems through a neural audio codec (e.g., EnCodec) and mixing them in all 2n combinations with real stems. This exposes the model to realistic partial-AI inputs and dense coverage of α values across [0, 1].

• Decouple content from artifact by using codec-reconstructed versions of the same human stems. This prevents the model from learning spurious content cues and forces it to rely on codec-specific artifacts, improving generalization to unseen AI generators.

• Use the regression formulation to recover proportional information from weaker artifacts. Because the binary detector’s output is miscalibrated (stays near 0 for α 0.9), a regression-trained model extracts signal from subtle “fakeprints” that binary classifiers miss — e.g., vocals and bass become partially separable only under regression.

• Add instrument-aware detection. Either run detection on separated stems before mixing, or condition the model on stem identity during training. This leverages the finding that drums/guitar carry strong artifact signatures while vocals/bass carry weak ones, improving overall accuracy on mixed content.

• Make detection window-length adaptive or multi-scale. The regression model improves with longer windows (MAE 0.076 vs 0.104, R2 0.85 vs 0.78), and per-window predictions enable temporal localization — useful for identifying which sections of a track are AI-generated.

• Report continuous calibration metrics (MAE, RMSE, R2, Pearson r) and boundary behavior. This exposes boundary compression (predictions flooring at 0.05 and ceiling at 0.95) and guides post-processing calibration to correct endpoint bias.

• Release the mixture-generation pipeline as a reusable library (ARIA) so other systems can generate controlled hybrid datasets — enabling reproducible benchmarks and fair comparisons of continuous detectors.

What the improved AI system can do:

• Given a mixed music track, output an accurate continuous estimate of the fraction of AI-generated audio energy (e.g., “45% AI”) instead of a binary verdict.

• Detect partial AI involvement in professional hybrid production, where stems (drums, bass, vocals, guitar) are individually replaced or generated.

• Maintain high accuracy at pure endpoints (>99% for fully AI or fully human) while also being informative in the previously unsupported middle range (α between 0.2 and 0.8).

• Localize AI-generated segments over time by aggregating per-window regression scores.

• Report instrument-specific AI burden — e.g., “the synthesized drums contribute most of the AI energy in this mix.”

• Generalize more robustly by training on combinatorially complete stem reconstructions, reducing content confounds and reliance on generator-specific artifacts.

AI-generated music is increasingly used at the stem level, with producers integrating synthetic drums, basslines, or vocals alongside human-performed instruments. However, current AI music detection systems are binary, treating tracks as either fully AI or fully human. In this paper, we reformulate AI music detection as a regression problem on a continuous AI energy ratio, alpha in [0, 1]. We propose a methodology that leverages a multi-track music dataset to assemble mixtures of human-performed and AI-reconstructed stems (obtained using a neural audio codec) with known proportions of each content type. Using this approach, we first show that a CNN-based model trained on fully AI-generated or human-performed tracks, which achieves >99% accuracy as a binary detector, when faced with mixed content, yields an output that rises with the AI stems' energy contribution, acting as a noisy and miscalibrated estimator. Our analysis of the influence of different stems shows that detection sensitivity depends on the instrument and reflects its frequency content: drums and guitar carry strong codec-artifact signatures, while vocals and bass are less detectable. Based on these insights, we train a similar CNN-based model for regression of alpha, achieving MAE = 0.076 and R squared = 0.85 on held-out mixtures from the same pipeline. These results suggest that the regression formulation is an initial promising step towards AI-music detection in realistic music production workflows.