Thanks for the detailed description. You say “the metronome follows the MIDI tracks. Therefore, the recorded audio is late compared to the metronome just like it is late when compared to the MIDI tracks.”
This is not how it’s supposed to be. Both visual and recorded audio should be in sync during recording and playback. This makes it a bit difficult to analyse. I am not 100% sure about external MIDI, but for audio playback, there should not be a noticable shift neither visual nor audible.
Could you try without MIDI? Just a click and let the Performer say “one two three four” or the like and see if there are still any shifts? Thanks for your patience.