DFU 64kB write size

[michalcourson]

The QSPI writes are fastest when everything aligns with the 64k boundaries. The FAT flash already writes in 64k chunks, but the DFU does not. I've added a buffer to defer the writes until either there is 64k to write, or the end of the file is reached.

I get a ~25% speed improvement with a ~3.5MB test file (41 seconds to 30) in the download portion of the flash procedure.

[stephenhensley]

Hm, so when I try to program my test program with this (rebased to the latest updates), I consistently get the same error, but at different parts of the program cycle.

The error is:

Download	[                         ]   1%        58368 bytes failed!
DFU state(10) = dfuERROR, status(11) = iString indicates a vendor specific error
dfu-util: Failed to write whole chunk: -1 of 1024 bytes

The failures occurred at various points: 40960, 58368, and 56320 bytes in three consecutive attempts.

Seems like it is consistently less than than the new block size, but I haven't looked much deeper than that yet.

[michalcourson]

hmm interesting. I did not see those errors, but I was also not rebased to the latest updates. I think I'll also have some time today to look into it

[michalcourson]

Errors are coming from here

DFUHandle::Result DFUHandle::Impl::MemoryWrite(uint8_t *src, uint8_t *dest, uint32_t Len)
{
    if (io_state.busy) {
        dfu_log.pushEvent(DfuLogger::Event::Type::WriteBusy, System::GetNow(), (uint32_t)dest, Len, 0);
        return Result::ERR;
    }
...

I'm pretty sure it's happening because the transfer size has changed from 4096 to 1024, so the timing is different. Looking at ways to fix this while still keeping it fast.

[michalcourson]

All I have so far is to extend the poll timeout, but that just slows the whole process down.

I think there are download speed improvements with the new middleware that I didn't notice anyway. I'm getting roughly the same speed during download with the current main branch that I did with this update. So I'm pretty content with just abandoning this idea.

However, we should also put "#define USBD_DFU_XFER_SIZE 4096" in the usbd_conf.h file as well to get the transfer size back up to where it was before. That does make a pretty big difference to the usb download speed.

[stephenhensley]

oh, I meant to include my test program to help with troubleshooting, but totally spaced.

But that makes sense to me, I can do that update to libDaisy's usbd_conf sometime early next week.

Feel free to close this if you'd like, otherwise we can leave it open until we see if the xfer size update in libDaisy results in sufficiently fast program times, and go from there.

[cristianpandele]

Sorry to resurrect a PR here, but I am a bit confused.

I am on the latest commit on main (8b279a8). This is from June 10th, so it must be same one you were using on June 12. If I try to flash a larger binary after successfully loading the following bootloader versions:

1. a version that I built from that commit, by running make from the ./bootloader folder
2. the "official" version in ./dist/dsy_bootloader_v6_4-intdfu-10ms.bin

(both of which are using the internal USB for flashing a binary)

I get two different behaviours:

1. Various errors of this sort, at different points, usually 0 or 1024:

Download	[                         ]   1%        1024 bytes failed!
DFU state(10) = dfuERROR, status(11) = iString indicates a vendor specific error
dfu-util: Failed to write whole chunk: -1 of 1024 bytes

2. Works fine

If I apply the changes suggested in this PR to 1. above, the failures move to somewhere above 60k, but it still doesn't work. Smaller binaries work fine.

I reset all my changes, as I wanted to see if I can recreate the dist binary included in he repo (2. above)... but no luck. Made sure to update submodules, the whole lot. All attempts ended up with a failure to flash the bootloader itself, like due to the version of dfu_util I am using complaining about the lack of USB strings:

dfu-util: Could not read name, sscanf returned 0
dfu-util: Failed to parse memory layout for alternate interface 0
make: *** [program-dfu] Error 74

Are you actually able to use the bootloader on the latest commit (with or without the changes in this PR) and flash binaries of large size (>200k)?

Please let me know if I should move this over to the Discord.

[stephenhensley]

@cristianpandele IIRC I did test flashing some larger files successfully with the current commit on main, but I can retest some time early next week to compare with your results.

Which version of the compiler are you using?

FWIW, the files in the dist/ were produced via the ./ci/build_release.sh on the pre-opensource, big declutter/cleanup commit (0e9ed56).
Those binaries can be recreated exactly on that commit using compiler version: (GNU Arm Embedded Toolchain 10-2020-q4-major) 10.2.1 20201103