After doing a bit of Googlin’ on your Audiogram 3, it seems others suffer from this problem too due to the fact that it apparently has some type of hardwired direct monitoring 
.
Is there any type of software control panel for the Audiogram 3 that might have monitoring options?
It’s explained in the Operation Manual. But, I would guess the option will be grayed out for you. It basically does what you are trying to avoid, allowing you to monitor your input signal directly with “zero” latency. Your hardware needs to support it for the option to be available.
You’ll have to do a little more research on that Audiogram 3. I might be wrong, but it looks to me like you might be screwed using that device.
Have you tried to contact Yamaha support? Steinberg support?
Otherwise, I think I’m out of ideas.