Ok, I spent about 1 hour playing around with this and watched the Dorico video about working with video in Dorico and markers. I have to conclude that they do stay attached to a specific frame in the video and so work as expected. Not sure why I could not get this working correctly before.
I created different markers in a short video with a short piano score.
I added tempo changes, immediately and graduals and inserted bars on different spots: The markers stay connected to the same video frame and the smtp numbers stay the same.
I moved a marker left or right which of course will move the other markers as well, but the smtp numbers in the markers stay the same.
So as far as I can see now, they do act correctly as scene markers, staying connected to the frames and not to the music.