Hi again Sabine!
Well, here's what I did...
I tried your configuration with 4 Quicktimes with the codecs mp4, H264, DV and AVID DNXHD120, and with an mp4 file containing an mp4 codec.
I connected my second monitor with a VGA and a DVI adapter. I would have tried HDMI as well, but I'm afraid my external monitor doesn't have a HDMI input.
In all 10 configurations, it worked well over here, so either it's a problem with the HDMI connector via MiniDisplayPort, if that's what you're using, or you're using some codec I didn't test, or it really is a Maverics issue, sorry, but I won't update to Maverics just yet, so I can't try that, and I can't imagine it's because of the .2 GHz less that your processor is clocking.
I hope that helps narrowing down your troubleshoot, I'm afraid that's all I can do over here.
If it's a codec issue I recommend using DV for SD and AVID DNXHD120 for HD video, I never had any trouble with those, as opposed to H264, P2 and AVCintra which have both given me quite a lot of grief, though nothing like what you have described.
Lots of luck, I hope you're successful, and don't hesitate to use the Steinberg Support channel, they really ARE very helpful!
HP Z800 Xeon 8-core 16GB RAM
RME Fireface UCX
BlackMagic DeckLink MiniMonitor SDI
MacBook Pro Core2Duo 2.8 GHz 4GB RAM