firmware update with sysex

[Dimitree]

Hi guys
I was wondering, how it is possible to update a firmware using Sysex messages? I know that many units do just that, for example M13, some midi footcontroller, and others.
do they use a sort of bootloader? the main program is translated in sysex first, then sent, and then translated again in binary code inside the uC?

[potul]

I don't know how commercial units do it, but I designed a MIDI footcontroller that can be updated via sysex. The idea is to use a bootloader that can receive data in MIDI sysex format, and write the program memory based on this. It is only a question of defining the protocol to use, and create the needed sysex message/messages.
In my case I took directly the hex file of the code, and group the bytes in groups of 16, something like

F0 00 XX XX XX XX ....  F7
F0 00 XX XX XX XX ..... F7

being XX the hex bytes.  And use a special combination to indicate the end.

The bootloader gets into "listen" mode when you power up the device and keep some buttons pressed. It then waits until receiveing sysex messages, writes the data to memory (allocating it in the correct place without overwriting the bootloader itself), and waits  until end of file received. Once done it resets.

I used as a starting point an existing bootloader that used serial communication and a PC software, and I modified it to use MIDI sysex instead.

Potul

[Dimitree]

many thanks! that's a great help

so basically you took the bytes directly from the hex and put them on the "data" bytes of the sysex?
then you need to write a bootloader that can handle MIDI protocol. it is right?
if so, my questions are:

1) if you don't press that buttons, the bootloader just load the main program inside the uC memory?
2) is the bootloader the one that will ignore F0 00 and so on, and the one who will understand when the file is completely received?
3) if you use Microchip PIC, could you post me some link where I could read more about where to write the data, how to define a bootloader, and so on?

many thanks again

[potul]

many thanks! that's a great help

so basically you took the bytes directly from the hex and put them on the "data" bytes of the sysex?
then you need to write a bootloader that can handle MIDI protocol. it is right?
if so, my questions are:

1) if you don't press that buttons, the bootloader just load the main program inside the uC memory?
2) is the bootloader the one that will ignore F0 00 and so on, and the one who will understand when the file is completely received?
3) if you use Microchip PIC, could you post me some link where I could read more about where to write the data, how to define a bootloader, and so on?

many thanks again

Well, you need to do some "coding" first. Each byte in the hex file is represented by a digit. I transform this into a sysex byte (ie: 02FF would become 00 02 0f 0f), and then add  a preceding F0 0A (0A is configured in the bootloader as the deviceID) and ending with F7. This is needed because midi bytes can only be up to 127, not 255 (bit 7 cannot be used). I used a simple Excel sheet to do the conversion. You can find it here:
https://sites.google.com/site/potulfx/apps/Midiboot_Hex2Syx.xls?attredirects=0&d=1
I would need to review,.... I don't remember if I implemented any CRC check. I think yes... because the hex file already contains some kind of crc, so I recall using it in the bootloader, but I can't confirm 100%.
The bootloader receives this data, erases all program memory except the bootloader and then writes all data.

Regarding your questions:
1)If you don't press any button at startup it just starts the normal program
2)Yes. The hex file last line has an indicator of being the last line. you need to look for it in the bootloader.
3) Here you can find it:

https://sites.google.com/site/potulfx/marketing-docs

It's the one named "midiboot.asm".

Some notes regarding it:

-The device I used needs that you delete memory in big chunks, so you will see there is a routing that deletes all program memory before starting to write.
-The bootloader takes care of reallocating the reset vector of the program so that it doesn't overwrite the bootloader one, and modifies a jump to make everything work OK.
-The bootloader is not able to program EEPROM, just program memory. Maybe in a later version....
-It uses "0A" as a device ID, so it will ignore any sysex with another device ID. You can change this easily in the program
-The bootloader receives each hex line in a sysex message. This is needed because writing to program memory is not fast, and you need to pause between each line to allow for the PIC to write. This means, you need to insert some waiting time on the PC side between message and message. I use MIDIOX for this, and you can configure this in the program.

Don't hesitate to ask for more clarifications.

Regards,

Mat

[Dimitree]

many thanks again for sharing all that, you're really kind!
now I study all those things and I see if I can do something good

[potul]

Well, you need to do some "coding" first. Each byte in the hex file is represented by a digit. I transform this into a sysex byte (ie: 02FF would become 00 02 0f 0f), and then add  a preceding F0 0A (0A is configured in the bootloader as the deviceID) and ending with F7. This is needed because midi bytes can only be up to 127, not 255 (bit 7 cannot be used). I used a simple Excel sheet to do the conversion. You can find it here:

There is a mistake in my previous sentence.... instead of "Each byte in the hex file is represented by a digit. I transform this into a sysex byte..." it should read:

"Each byte in the hex file is represented by 2 digits. I transform this into 2 sysex bytes..."

So, each pair of characters in the hex file is coded into 2 sysex bytes, as shown in the example. You can see the details in the excel file.

Regards,

Potul

[Dimitree]

hi Potul, I noticed you used a PIC16F88. Is the whole 16F series able to self-programming? Microchip specifies this features not on every device

[potul]

no. not all devices support self-programming. You will have to take a look at the datasheet or the information in Microchip website. I know for sure 16f88 can do it, but don't know about the rest.