Depends on the platform.
Arduino I/O is designed around the hardware in the MCU so it doesn't present so much pin contention. When you start to use library code, then you need to be careful as it can then have to use a standard I/O pin for another purpose.
I wonder is the controller has an active USB connection? This can carry a 1kHz whine due to polling every 1ms (the bane of cheap soundcards). Having USB in an audio system obviously can be done, but needs better filtering.
If you want USB for a debugging serial monitor of your code, all might be well when you take that out and disconnect the USB.

This looks ok to me. One thing I don't see is a larger cap on the regulator output. I'd fit at least 100uF in addition to the 100nF (keep that). Regulators can be unstable without enough capacitance (should be some on the regulator input too).