rolling back the volume is fine, but you need to wire it a volume, not as a variable resistor. connect your 386 to the output cap to the "hot lug, CW" of the pot, ground the "cold, CCW" lug, and take the output from the pot wiper. this will then present a [reasonably] stable and [relatively] constant load of 100k to the 386, whatever volume setting you choose, instead of a varying load resistance as in yr dia.