Bootloader/ring_buffer questions

Hey everyone, would appreciate some help on this matter:
I’m trying to build a midi bootloader for an stm32f103 so I’ve looked through a lot of the code on the yarns bootloader since it’s f103 based if i’m not mistaken.
I’vre previously managed the simple bootloader where the program to be flashed is in an array in the bootloader program, and that works perfectly, so I wanted to take it to the next step and be able to send the firmware in sysex format through midi.
If you look at my code it’s very similar to the yarns bootloader program currently and I also have an oled screen for a visual representation of what’s going on. what I’ve figured out so far is that the ring buffer (stmlib/utils/ring_buffer.h) function “readble()” is never true and thus skips to jumping to the main program. I cannot understand why this is happening though.
The logic behind it, if i’m not mistaken, is wait for midi input, parse it through the buffer, check if it’s sysex and correct, start copying from the buffer to flash memory and when it’s finished -> jump to main firmware.

If sample code is needed I can post.

It would be quite weird to invoke a function if this function always returned false!

I think the code makes the core logic of the bootloader quite obvious:

First we check if the encoder is pressed. If it is not, we directly exit the loop and jump to the application (main) code.

If the encoder is pressed, we enter the main loop. The condition to remain in this loop is:

midi_in_buffer.readable() && !exit_updater

ie, as long as there is MIDI to parse and we are not done with the update.

1 Like

Adding the outer while loop for exit_updater helped not jumping to firmware, however, you misunderstood.
I did not mean that the function by design always returns false, of course…
What I meant was that in my application I must have done something incorrectly and the ring buffer doesn’t seem to be available at any point. For example right now it is running constantly in that outer while loop waiting for the function to return true for it to enter the inner while loop and start copying to flash etc.
To be honest, I’m still trying to completely understand the ring_buffer code so I can be able to debug and figure out what I should be doing.
Thanks for the reply.

As a side note, one reason I might be having a hard time with understanding some of your code is because I’m currently using the HAL library (because of cubeMX). Do you think it’s a waste using this library and software?

So you have to check that some bytes are actually put in the ring buffer somewhere else. This can be done either by handling an interrupt raised whenever the UART receives a byte, or by periodically polling (say every 0.125ms) the UART. The former approach is how I do it:

Note that this code is in the interrupt handler for the SysTick timer, and that it is configured to run at 8kHz.

You also need to check that the UART is correctly configured, that the related peripheral clock is running, that the GPIO pin is configured with the right alternate function etc.

I got into STM32F programming 8 years ago when HAL/CubeMX didn’t exist. I have never looked into it, but I doubt that’s the cause of your issues.

Hmm UART is configured properly and does work, I checked earlier with a simple note-on / led on-off. I’ll have a more in depth look at the systick & buffer polling part, maybe i’ve misconfigured something there or maybe something slipped my attention.
I hope it’s something like that which would prove easier to fix rather than scratching my brain trying to debug the rest…

Update: So I managed to get the bootloader working great, however I now have an issue with the .syx files.
I’m using the hex2sysex tool and although some hex files convert great, I have some larger ones that don’t match the corresponding hex files, which is really weird. Is there a problem with larger files?

Can you post some examples here?

the first 200 bytes or so in the hex file are as follows:

6000 2000 01E5 0800 31CD 0800 31CF 0800
31D1 0800 31D3 0800 31D5 0800 0000 0000
0000 0000 0000 0000 0000 0000 31D7 0800
31D9 0800 0000 0000 31DB 0800 31DD 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
33A9 0800 33A9 0800 33A9 0800 31E9 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
31F9 0800 33A9 0800 3209 0800 33A9 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
3219 0800 3229 0800 3239 0800 3249 0800
33A9 0800 3259 0800 33A9 0800 3289 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
3299 0800 33A9 0800 3279 0800 3269 0800
33A9 0800 33A9 0800 33A9 0800 33A9 0800
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
F85F F1E0 B508 F000 F87F BE00 E7FE F01E
0F04 BF0C F3EF 8008 F3EF 8009 4671 4A37

but when i use the hex2sysex tool it becomes like this:

F0 00 21 02 00 66 7E 00 0A 0F 04 09 00 00 00 00 00 0C 04 0A 00 00 00 00 06 09 04 0A 00 00 00 00 0C 05 04 0A 00 00 00 00 02 02 04 0B 00 00 00 00 07 0F 04 0B 00 00 00 00 0D 0C 04 0B 00 00 00 00 03 09 04 0C 00 00 00 00 09 05 04 0C 00 00 00 00 0F 02 04 0C 00 00 00 00 04 0F 04 0D 00 00 00 00 0A 0C 04 0D 00 00 00 00 00 09 04 0E 00 00 00 00 06 05 04 0E 00 00 00 00 0C 02 04 0E 00 00 00 00 01 0F 04 0F 00 00 00 00 07 0C 04 0F 00 00 00 00 0D 09 04 0F 00 00 00 00 03 05 05 00 00 00 00 00 09 02 05 00 00 00 00 00 0E 0F 05 00 00 00 00 00 04 0C 05 01 00 00 00 00 0A 09 05 01 00 00 00 00 00 05 05 02 00 00 00 00 06 02 05 02 00 00 00 00 0B 0F 05 02 00 00 00 00 01 0C 05 03 00 00 00 00 07 09 05 03 00 00 00 00 0D 05 05 03 00 00 00 00 03 02 05 04 00 00 00 00 08 0F 05 04 00 00 00 00 0E 0C 05 04 00 00 00 00 04 09 05 05 00 00 00 00 0A 05 05… etc. etc.

where as in another smaller program the bytes in the hex file and the bytes in the sysex files match up (taking into account the 00 inserted)

for example:

6000 2000 81E5 0800 8EFD 0800 8EFF 0800
8F01 0800 8F03 0800 8F05 0800 0000 0000
0000 0000 0000 0000 0000 0000 8F07 0800
8F09 0800 0000 0000 8F0B 0800 8F0D 0800

becomes this:
F0 00 21 02 00 66 7E 00 00 00 06 00 00 00 02 00 0E 05 08 01 00 00 00 08 0F 0D 08 0E 00 00 00 08 0F 0F 08 0E 00 00 00 08 00 01 08 0F 00 00 00 08 00 03 08 0F 00 00 00 08 00 05 08 0F 00 00 00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 07 08 0F 00 00 00 08 00 09 08 0F 00 00 00 08 00 00 00 00 00 00 00 00 00 0B 08 0F 00 00 00 08 00 0D 08 0F 00 00 00 08

I could be wrong but it seems that although jumbled up, all the bytes match and are in there, but in the first case it is not…

the only options i use in the python script is setting product id to 66 and page size to 1024
I noticed this because while programming via midi/bootloader and looking through the flash memory, the bytes were not correct so I went to look at the sysex data and there I saw this mismatch.
I could be wrong of course, maybe it’s something normal? I don’t know

Can you post the actual hex and syx files?

test.zip (103.5 KB)
here you go, I think that for some reason it’s not converting all of the data and is omitting some of the earlier bytes…

It’s because the .hex file reader I wrote ignores the “Extended Linear Address” field of the HEX file, ie, it’s tailored for files under 64k (simply because I have never developed any MIDI upgradable device with more than 64k of code).

Aha, so that was the problem!
How do you reccomend I go about this?
Should I mess around with the python scripts then?

I’ve modified hexfile.py to handle such files now.

3 Likes

Wow, thank you so much Émilie, you are amazing! It works perfectly!

1 Like