How are you powering the device? Do you have pedals or anything sharing the same power? I've had some digital pedals that caused noise just by sharing the same 9v daisy chain, with no audio cables plugged in.
Regardless, you can star ground the digital, analog, and chassis grounds back where the power supply comes in for starters, if you haven't already.
Or, isolate the grounds completely? Since you aren't processing the audio, just switching it, you could easily keep the grounds totally separate. You can make the chassis part of the audio ground (like it is a big bulge in the cable shield), and tie the digital ground alone back to the power supply (-).
If that doesn't help and you think the interference is electrostatic (the audio wires being near noisy digital signals) rather than grounding, you can make little grounded tin hoods to shield the analog sections from the digital sections. Also shielded cable for the audio could help as well if you have runs longer than a couple inches.