I’ve tried that (with and without rebooting) and it makes the situation worse.
It is definitely much better to have ASIO-Guard ON (it reduces the Average Load and Real-time Peak readings dramatically), although it is not enough to just turn ASIO-Guard On - ASIO-Guard needs to be activated for each plugin on the Plugin Information screen (N6 does not always do it automatically).
I have found that Freezing the VST Instruments appears to be the best solution, so far.