Hey El, this is where it's handy to have a switchable FX loop that turns the signal off (wet or dry or mix of the two).
I have a related situation in my live rig as I've started to use the IPS33 smart shift again, so I feed it from send 3 and return down 2 channels which I then send to main mix and also L to send 1 and R to send 2 (which are Quadverb L/R ins) so when I select a bypass program on IPS I get increase in dry level both in main mix and QV. But I can balance this out.
If you are getting phase issues, you can try inverting the phase of the delay output (but this will also affect the wet delay signal (which you are happy with "perfect")). Alternately you could feed the delay output through an on/off switch, leave the delay wet but switch the signal off when not using it. (not a very elegant solution though is simple). More expensive mixing consoles have a phase reverse switch on each input, also some DI boxes have similar. But test by reversing the +Ve -Ve connections on one end of the lead you use to connect the delay output. If that fixes your problem (and doesn't wreck the wet delay) you could look for a phase inverter. You can by adapter plugs which are wired to phase reverse, probably the easiest way to test without having to re-solder a lead.