I‘ve been thinking about your case - and I should have been thinking earlier 
Using external MIDI can‘t work, because it is beeing triggered by your local playback engine, while other playback sent to the Performer is beeing triggered way ahead of that (internal playback sent to the Performer when you hit play, but which you don‘t hear; local playback therefor starts after a second or so depending on Remote Delay while Performer data have already arrived). In general, the Performer can not play to what you actually hear, that would be jamming, which VST Connect doesn‘t support.
So you will have to record your external MIDI to an audio track beforehand, then all should be fine.
Good luck!