I think the most appropriate solution is to have a “control room” output for the reference track. (Does not exist!!) With that you can use monitor switching and headphone handling. I use a extra pair on my soundcard and use a hardware monitor switch. From a daw point of view that is silly, but it is only working way for me.