It makes much more sense to me to use send levels as you would in a room-to-room recording situation. Only then can one provide an individual mix to the Performer. Routing to the “To Performer” group channel directly (multi-output routing) will work too, but without the benefit of individual levels.
If the dropouts only appear when playback or record is engaged, you should raise the “Remote Latency” setting in the settings page. Usually around one second is ok, if you have to go much higher for seamless operation your connection isn’t really good (high “ping”).
If the dropouts appear all the time (even in “chat mode”, that is, when no playback is active), then you should lower the upstream bitrates for audio and video transmission. If you can’t go lower and still have problems, make sure you use a solid ethernet cable to your router instead of WiFi, and if that doesn’t help either, you may consider to change your IP provider 