Pitch detection

[potul]

Hi,

I've been doing some research around pitch detection algorithms, that I plan to use in  a couple of projects, including a Guitar-2-midi converter using a dsPIC.

I have compared Autocorrelation, STFT and HPS, and done some testing in MPLAB. What I've found is that for a given sampling rate and block size:

-Autocorrelation works best on low frequencies
-STFT works best on high frequencies
-HPS is great to distinguish the fundamental frequency from the harmonics.

Which is not surprising, because:

-Autocorrelation resolution depends on the sampling rate. The closest frequencies you can distinguish are separated by a period of one sample, so for low frequencies (long periods), the error is smaller.
-FFT resolution is constant in hz, but frequency differences between notes are much higher for high frequencies, so the errors are smaller.


So, I decided to put a hybrid algorithm, something like this:

Step1: Perform a first  FFT
Step2: Do an HPS and calculate the fundamental frequency. At this point resolution is very bad, and determined  by the FFT bins width
Step3a: If the fundamental frequency is low, perform and Autocorrelation. This will give us the frequency with a much higher resolution
Step3b: If the fundamental frequency is high, perform another FFT with an overlap to the first one (it would be the next frame in the STFT). With this, calculate the phase difference for the fundamental bin and apply the necessary correction to compute the real frequency. This also gets us to a much better resolution than the FFT bins.

I've implemented the code and tested it in MPLAB (didn't have time to test in a real PIC yet), using real guitar samples, and the results are promising. But I have a doubt regarding the latency.

The autocorrelation takes around 4 ms to compute and the FFTs around 1.5 ms. This looks acceptable, but what concerns me is the size of the block needed.  I would assume that in order to be able to track the lowest frequencies I need to analyse at least a block corresponding to the period of the lowest note. This is around 12ms for the low E. But for the autocorrelation to work I think I need more, as I need to detect repetitions of the signal.
By doing some experimentation I've found the optimal to be around 25 ms.
So, this would mean a latency of around 30ms just for the sampling and autocorrelation, and probably I would need to add some more computation. But I see that the total latency would be around 50ms or so.

Would a latency of around 50ms be too much for a guitar-2MIDI device?
Do you have any experience with autocorrelation?

Mat

[Hides-His-Eyes]

50ms is too much latency for recording. But a synth sound might easily have an attack of 100ms. So it really depends what it's being used for.

[Taylor]

A programmed pitch-detection PIC would be a very popular DIY product IMO! I'd buy some for sure, and once a project or several arose using it, it would be a hot item. (Sorry I have no useful advice for you).

[potul]

Ok, I'll give it a try with the 50ms latency and see how it feels. Now I'm 3 weeks out on a trip an can only work in the simulator, but once I get back home I'll try to burn it and see how it feels.

Taylor, regarding the pitch-detection chip, I wonder what applications could have... Once I finish my guitar-2-midi project, it should not be complicated to remove the midi functionality and add something to make it work more generically.
I wonder how it would be convenient to interface other devices, outputting a CV? Or maybe generating a clock signal? What do you envision?

[cpm]

what about parallelizing the computations, so when the first FFT is done you already have the previosus results from the different low and high methods, even if they are from previous samples.
If enough memory and processor is available you could even do overlapped calculations. Instead of doing 1 20ms correlation algorithm at a time, perform 4 (still 20ms each one) at each 5ms tap, and apply some heuristics over the results to better aproximate, and get less latency and less glitches.

does it make any sense?

regarding a general purpose module: both a CV (pwm) and a square output locked to the detected frequency would be excellent

[CynicalMan]

I hate to be a downer, but Distortion with lots of low-pass filtering generally do pretty well at monophonic pitch detection. Play a Big Muff with Sustain set high. That gets the fundamental as a square wave pretty easily, and it doesn't get fooled by harmonics. Now polyphonic pitch detection would be sweet. Getting a chip that could track polyphonically would open up the door for a huge range of fun effects.

[Taylor]

I hate to be a downer, but Distortion with lots of low-pass filtering generally do pretty well at monophonic pitch detection. Play a Big Muff with Sustain set high. That gets the fundamental as a square wave pretty easily, and it doesn't get fooled by harmonics. Now polyphonic pitch detection would be sweet. Getting a chip that could track polyphonically would open up the door for a huge range of fun effects.

Something along those lines can indeed be good signal conditioning before pitch detection (although filtering and squaring can be done much more easily in software than having to build a Big Muff), but how does that equal pitch detection? All the detection would still need to be done after that point to get a CV, varying resistance, or any controller based on the pitch of the signal.

Taylor, regarding the pitch-detection chip, I wonder what applications could have... Once I finish my guitar-2-midi project, it should not be complicated to remove the midi functionality and add something to make it work more generically.
I wonder how it would be convenient to interface other devices, outputting a CV? Or maybe generating a clock signal? What do you envision?

Outputting a CV would indeed be magical. One could take that and then do whatever else with it fairly easily. If there was a way to get a precise varying resistance, that would be cool too, as the monolithic function generator ICs like the XR2206 need a resistance for pitch control as I recall. I suppose a digipot would be one solution if there was integrated feedback from the digipot so the PIC could make up for tolerance issues in the digipots. There would be that zippering effect due to limited digipot resolution, but I think this could be smoothed, or looked at as a feature, like a built-in microtonal Auto-Tune.  

Here are some apps off the top of my head, for which I think people would be excited to use this:

-guitar/bass synths (if a simple PIC circuit interfaced with the XR2206 you could have a very tiny, and very capable synth)
-pitch tracking filters
-pitch tracking ring modulators

[slacker]

Taylor, the XR2206 can be voltage controlled, there's a design by Thoms Henry here http://www.birthofasynth.com/Thomas_Henry/Pages/XR-VCO.html needs a bipolar supply but other than that it looks a fairly straightforward build.

A pitch to CV chip would be great, I'd imagine once you've done the pitch to midi part the CV would be fairly trivial. Look forward to seeing how this progresses.   

[CynicalMan]

I hate to be a downer, but Distortion with lots of low-pass filtering generally do pretty well at monophonic pitch detection. Play a Big Muff with Sustain set high. That gets the fundamental as a square wave pretty easily, and it doesn't get fooled by harmonics. Now polyphonic pitch detection would be sweet. Getting a chip that could track polyphonically would open up the door for a huge range of fun effects.

Something along those lines can indeed be good signal conditioning before pitch detection (although filtering and squaring can be done much more easily in software than having to build a Big Muff), but how does that equal pitch detection? All the detection would still need to be done after that point to get a CV, varying resistance, or any controller based on the pitch of the signal.

My point is that using FFTs isn't necessary for monophonic pitch detection. Clipping and LPFs will get you a square wave, then you can just count the pluses for the pitch. Going from there to a CV is pretty easy. FFTs are nice for polyphonic pitch detection, but then we're looking at a few CV outputs instead of one pin. Not that I wouldn't love that. 

[toneman]

Well, if U are going to use a basic processor, U will still could use an "analog front end".
I like PICs, but if U want to use the DSP PICS, the following will not apply.

If U want pitch to voltage, the National LM2907 has been out for a very long time.
Used mainly for tachometers, it's a good linear frequency to voltage interface.
see here:
http://www.national.com/mpf/LM/LM2907.html#Overview

U probably could still use a good fundamental extractor/front end.
EH uses a "self-adaptive-threshold detector" in its Octave Multiplexer.
It's the parts just before the 4013.  The 4013 is the octave down maker.
Basically a flip-flop, divide-by-two thingie.
see here:
http://topopiccione.atspace.com/pjimages/EHOctaveMultiplexer.sch.gif

A while back, back in 2006, markusw and I and a few others were looking for an octave up to use with the
RingStinger clone.  We found several XOR-type circuits that all had capacitors in them.  Only worked over
a very narrow range of audio frequencies.  What we needed was an octave-up that wasn't capacitor-based.
I ended up getting a couple of cheap Behringer pitch shifters that did 2 octaves up AND 2 octaves down.

Good luck with your research!

[ChrisA]

[
Something along those lines can indeed be good signal conditioning before pitch detection (although filtering and squaring can be done much more easily in software than having to build a Big Muff), but how does that equal pitch detection? All the detection would still need to be done after that point to get a CV, varying resistance, or any controller based on the pitch of the signal.

The modern state of the art method is NOT to measure frequency.  You measure PERIOD and to get more accuracy you can average them.    This can also track "bends".

The way to measure period is first to "condition" the signal, detect the zero crossings.  This is pretty much the same this as "squaring"  Either way now you have a digital square wave

Next you need a fast counter.  Typically this runs 10MHz but it could be as slow as 1MHz for work with guitar.   The raising edge of the digital signal gates the clock to a counter, the next raising edge stops the counter.  Now you now the period.  If we assume a 1K audio input and a 10MHz counter we know the frequency to better then 1 milli-Hz and it only took one milli second.  Keep a running average of these counters.    Actually this was "star of the art" and therefor WAY to expensive to use in a stomp box in the 1970's.  Now a 1MHz is very easy and cheap to build.

The FFT method requires that you sample the signal for many cycles and then the frequency resolution is only as good as the block length.  You trade latency for resolution.  Period measuring give you any desired resolution in only one cycle and requires no memory or A/D conversion.

I have a device on a 4x5 inch PCB that costs about $50 to build that measures period at at the sub-nanosecond level.  That is gross overkill for this application.  You could live with 1000X less performance. So it should cost a LOT less then $50.

[potul]

The modern state of the art method is NOT to measure frequency.  You measure PERIOD and to get more accuracy you can average them.    This can also track "bends".

I already have a PIC based device that uses zero-crossing algorithm to track period, but I'm not satisfied with its performance, that's why I'm trying different approaches. I do have some limits due to the uC, I can't be sampling at 10Mhz.

The FFT method requires that you sample the signal for many cycles and then the frequency resolution is only as good as the block length.  You trade latency for resolution.  Period measuring give you any desired resolution in only one cycle and requires no memory or A/D conversion.

With an STFT I can get much higher resolution than the bin width using the phase difference information, but it is true that the bigger the FFT block the better. Whith autocorrelation I get exactly the same resolution I can get with zero-crossing, and it only depends on the sampling frequency. The problem I have is about the size of the sample I need to get results.

Anyway, your proposed method also has  a similar problem than the one I'm facing, because in order to get the period you need to wait at least one full cycle (around 12ms for a low E), and if you need to do some averaging you might need some more time.
Using autocorrelation I need between 1.5 and 2 periods of signal to get a reliable period for the lowest notes., which is not far away of the zero-crossing latency.

Mat.

[~arph]

I hate to be a downer, but Distortion with lots of low-pass filtering generally do pretty well at monophonic pitch detection. Play a Big Muff with Sustain set high. That gets the fundamental as a square wave pretty easily, and it doesn't get fooled by harmonics. Now polyphonic pitch detection would be sweet. Getting a chip that could track polyphonically would open up the door for a huge range of fun effects.

How doesn't it get fooled by harmonics? for a narrow range yes, but not over the full range of frequencies (especially the low end)

[CynicalMan]

In my experience, Big Muffs do a pretty good job with the low end. They don't have the issue that some fundamental detectors do with switching between a note and its second harmonic. If you hit a low E on a Big Muff, it stays there. YMMV.

[Gurner]

I hate to be a downer, but Distortion with lots of low-pass filtering generally do pretty well at monophonic pitch detection. Play a Big Muff with Sustain set high. That gets the fundamental as a square wave pretty easily, and it doesn't get fooled by harmonics. Now polyphonic pitch detection would be sweet. Getting a chip that could track polyphonically would open up the door for a huge range of fun effects.

How doesn't it get fooled by harmonics? for a narrow range yes, but not over the full range of frequencies (especially the low end)

agreed, the 2nd harmonic is especially strong on the lower frets/open strings.....it crosses zero, which completely screws traditional zero cross pitch detections.

[Ikedude]

I ended up using a modified AMDF for my system like this. I got a little ambitious and built humbucker sized pickups so I could track each string individually. Here's a pic:



Don't be fooled by the USB cable in the background, it just has MIDI output, ha.


For more info on the AMDF here is a paper: http://www.asel.udel.edu/icslp/cdrom/vol2/218/a218.pdf

Basically compare the signal to a time (sample) shifted version of itself. A guitar is A LOT easier than a general purpose signal because you have a limited number of possible values (frets 0 - ~24). This algorithm uses subtraction and absolute-value, a little less computationally expensive than a full-blown FFT. I found accuracy was good for as little as 1.5 cycles and the computation time was on the order of ~2.5 ms. So delay for low E is around 20 ms.

I ran into problems with interference on the guitar and haven't quite figured out the best shielding arrangement. In the meantime I mounted this one on my bass:



Main problem here is the period length, lag on the order of ~30 ms is very noticeable. I was trying to figure out a good way to do a half-cycle AMDF... no luck yet, ha.

I used a PIC24FJ16GA002 at 32 MHz to do all the sampling, computation, and communication. Other circuitry is just to filter and shift the AC signal.

I'd be happy to share the code but I don't think its entirely legible, ha.

[Taylor]

Googily moogily. I want one for bass. Any chance you'll make boards available?

[Ikedude]

I ordered the boards through Golden Phoenix so I have way more than I would ever use. If you're comfortable doing SMT soldering (I'm not so I used stencils and solder paste) and have a way to program PICs (the five pins are for a PICKit), then we could probably figure something out. I got busy lately and haven't had the time to mess with it, so its still pretty rough around the edges.

I was motivated to build this to try and use it with Rock Band 3's Pro Mode, so right now its more of a fret-detection device than it is a pitch detector.

[zeptin]

Well, I for one would love to see the code and board schematic

I assume the cylindrical objects are the pickups you made, how are they constructed? Pickup wire wrapped around small magnets? How are they mounted on the board?

I have a lot of other questions, but maybe the code will clarify some of them

[Ikedude]

I don't really have a place to host the code. I'm not sure I ever did a schematic, but here is the layout.  Its a lot easier to find a place to host pictures than anything else, haha.



At first I was going to wrap my own coils but that didn't work out very well. Without a dedicated winder its really hard to keep good tension on magnetic wire around 3/16" steel rods.

I ended up using these for the guitar:
http://search.digikey.com/scripts/DkSearch/dksus.dll?Cat=196627&k=M10029-ND

I needed a lower profile for the bass since I wasn't gonna drill a pocket for tests:
http://www.newark.com/bourns-jw-miller/rl262-153j-rc/choke-15000uh-20ma-5-0-5mhz/dp/86K3337?Ntt=RL262-153J-RC

If you look at the datasheets for those you'll see they have a little pocket on the bottom. This was the perfect size for these:
http://www.kjmagnetics.com/proddetail.asp?prod=D21B-N52

It was a lot easier to mount the inductors than to try and figure out a way to mount some steel rods. There are two coils for each string wired in series. Like a humbucker they are reverse-wound (in this case that means I rotated them 180 to swap the leads) and the magnets on the bottom have opposite polarities facing up. They are pretty quiet but also sensitive to multiple strings on the guitar since they are so close. Maybe a strip of steel between each pair of coils would help isolate them, but I'm not too good with magnetics so I'm not sure.