Okay, I've dusted off the Elk Pi and Blackboard and actually got it working in the last couple days. It was one of those situations where there is actually not enough information online to succeed. I had to get help through their forum, which fortunately only took a day or two to figure out what was going on.
The next step will be for me to get some of my Faust code I'd done for the ESP-32 and try to make a headless VST3 out of it. The Elk people did do a "Faust on the Elk" seminar at CCRMA earlier this year... early enough that such things were still going on. CCRMA is only about an hour from here, I should pay closer attention to what they are doing!
https://ccrma.stanford.edu/I had previously used TouchOSC with the ESP-32 to control FX from an Android tablet. The Elk OS also supports OSC and Ableton Link so I'm going to try to get some tempo sync'ed delay action going on. I first tried that with my Eleven Rack and IMO it makes a huge difference in terms of how useful the delay seems, beyond a little slapback for ambience. Elk uses a "Sensei" layer to provide user input from pots, switches, sensors, etc. and there are some examples already prepared for the setup on the Blackboard. I really doubt that I am going to add my own sensors, but who knows?