Here is an experiment of audio-to-audio time alignment of classical music recordings based on the method of: “Invariant Audio Prints for Music Indexing and Alignment”, see https://anasynth.papers.ircam.fr/2024/CBMI/ .
First, different recordings of the same orchestral works (from Debussy and Mahler) conducted by different conductors (Boulez, Karajan, ...) and played by different orchestra (Wiener Philarmoniker, Lyon National Orchestra, ...), are analysed by pairs to estimate the time alignment mapping. Then, the time of one recording is modified to be synchronized on the time of the other.
Second, the same test is done with pairs of recordings from the same music work, but with different arrangements, and so different instruments. For example the fabulous Debussy's "Pagodes" music piece is analysed from the original piano version of Claude Debussy, then the orchestral version of André Caplet, and also the arrangement of Percy Grainger (for tuned percussive instruments and reed organ, with timbre similar to the Gamelan orchestra, the original Debussy's inspiration).
For each music work and movement, a list of original recordings have been downloaded (from youtube). Previews of these original recordings are available in the first tables "Original recordings".
Then, using pairs of these original recordings, the audio-to-audio time
alignment is estimated. The second tables "Realigned pairs"
propose a preview of the realignment:
"The sound signal of the recording B of the pair is modified in order to
synchronize its time on the time of the other recording A."
Then, the two signals are merged into a unique stereo sound file:
When the cursor is over a realignment audio preview, a figure appears on the right of the table: it displayes the estimated time alignment of the two recordings of the pair A vs. B. With the time of A on the x-axis, the blue curve presents the delay of the time of B compared to this of A. For example, if at x=100sec the blue curve has the value y=10sec, it means that the chord played in A at the 100th second, is played in B at the 110th second, then B is delayed. Positive values correspond to a delay of B, negative values correspond to an advance of B compared to A. The slope of the blue curve informs about the speed of B compared to A: positive slopes correspond to lower tempi of B (the delay grows), and negative slopes correspond to higher tempi of B, compared to A. The green curve presents also the diffence of tempi between A and B, using the estimated time stretching factor expressed in cents. "0 cent" corresponds to identical tempi, "100 cents" correponds to a tempo of B which is twice lower than this of A (at the same corresponding time in the music work), and "-100 cents" corresponds to a tempo twice higher.
As a general conclusion about the results, presented below, the time alignment is very good for orchestral recordings. The error of this time alignment (delay between the left and right channels) is usually imperceptible. In some rare cases we can hear significant delays, lower than 1 second.
Nevertheless, when aligning 2 recordings with at least one solo instrument (piano, harp, guitar), the time alignment almost works, but with a lower time accuracy. Even if the time alignment succeeds to roughly follow the two recordings, the time error is clearly audible most of the time. Two reasons can be the causes of this lower accuracy:
We here test the time alignement of 3 recordings of the marvellous "La Mer" of Claude Debussy. The 3 recordings are from:
We here test the time alignement of 2 orchestral recordings of the fabulous "Pagodes" of Claude Debussy. This work, initially composed in 1903 for piano as the first movement of the "Estampes" suite, was arranged for orchestra by André Caplet in 1923. Below is another test with time alignment with piano and other arrangements. The 2 recordings are from:
We here test the time alignement of 2 orchestral recordings of the famous "Clair de Lune" of Claude Debussy, from the "Bergamasque" suite (3rd movement). Again, this work was composed in 1890 for piano, and it was arranged later for orchestra by André Caplet. Below is another test with time alignment with piano, harp, and guitar. The 2 recordings are from:
We here test the time alignement of 2 recordings of the extraordinary 3rd symphony of Gustav Mahler. The 2 recordings are from:
We here test the time alignement of 3 recordings of the tragic and splendid 6th symphony of Gustav Mahler. The 3 recordings are from:
Remark: at the very begining of the alignment of Abbado to Rattle, during few second, the alignment is not perfect and the delay is audible. But the time is correctly synchronized after few seconds.
We here test the time alignement of 2 versions of the two wonderful "Arabesques" of Claude Debussy. The first recording is from the initial composition for piano, and the second recording is from a version for harp, so with a different instrument. The piece contains two movements, and the two recordings contain the two movements. The 2 recordings are from:
Attention:We here test the time alignement of different versions of the famous "Clair de Lune" of Claude Debussy, from the "Bergamasque" suite (3rd movement). The 5 recordings are from:
We here test the time alignement of different versions of the fabulous "Pagodes" of Claude Debussy. Let's remind that this work was initially composed in 1903 for piano, and it was arranged for orchestra by André Caplet in 1923. Then in 1928, Percy Grainger did a second arrangement for tuned percussion instruments (celesta, xylophones, marimba, tubular bells, vibraphones, glockenspiel, gong, and dulcitone), piano, and reed organ. Note that even if this second arrangement uses instruments common in the occidental music, the timbre are close to the Gamelan instruments (Indonesia), and it is well known that Debussy got inspiration from this oriental music. So, we tested the time alignment with: