I confirm what molo is saying, that clicking directly on the VST keyboard does not have the “gradual latency” issue.
Seeing as we all have Steinberg interfaces, and how I did not have this problem three months prior with an SPL Crimson, seems pretty clear that it’s an issue with the Steinberg driver somewhere.