nRF52840: MCU to MCU updates via activation pin and CircuitPython filesystem programming #283

m-mcgowan · 2022-10-21T17:20:59Z

Overview

This PR implements changes in the bootloader to enable MCU to MCU updates, in a similar manner to the built-in ROM bootloaders provided on STM32L4 and ESP32 devices. The host MCU can programmatically enter bootloader mode by asserting a DFU pin on reset, which places the device in DFU bootloader mode, with the DFU protocol running over UART rather than USB serial.

Additionally, the bootloader protocol has been extended with commands to allow erasing and writing to QSPI flash. This can be used to flash a pre-prepared FAT filesystem image to QSPI flash, such as for use with CircuitPython, enabling factory programming of CircuitPython scripts along with the CircuitPython runtime.

All changes are opt-in only and are intended to be fully backwards compatible, meaning no changes to boards that don't have the new configuration flags defined.

Configuration Changes

Boards that support DFU activation via a pin should define DFU_PIN_ACTIVATION=1.

The button pin, DFU activation pin and FRST pin configurations now specify a pull direction to allow active high or low depending upon how that GPIO is used. The defaults reflect was was previously being used (BUTTON_PULL).

New optional defines PIN_DFU_ACTIVATE and PIN_DFU_ACTIVATE_PULL allow a particular board to define the DFU activation pin and pull direction. When these defines are present, DFU pin activation is compile-time enabled. These defines are present only for the Adafruit nRF52480 express board, which uses BUTTON_2 - feather pin D2. This was previously the FRST signal, but factory reset functionality has been removed prior to this PR, and so it was safe to reuse this.

The define DFU_EXTERNAL_FLASH enables the protocol extensions for programming the QSPI flash. It is defined as 0 by default, and defined as 1 only for the adafruit_nRF52480_express board.

When DFU activation pin is enabled, both USART and USB CDC serial are both compiled in, and selected at runtime. When the DFU pin is not asserted, the bootloader functions as it did before - USB CDC and MSD is enabled for use with adafruit-nrfjprog and UF2. UART is still used by default for boards with the MCU subvariant nrf52, as was the case prior to this PR.

Other Implementation Details

I added some critical sections around the hci memory pool and hci transport code, to prevent race conditions accessing shared state between the main program thread and code running on UART interrupts.
When DFU is started via the GPIO, the device is not restarted when the application is flashed, in case the MCU wishes to flash more binaries, such as the QSPI filesystem. bootloader_app_start() previously both registered the app and launched the app. The function has been factored out into functions so the app can be registered without necessarily starting it.

Testing

The code has been tested extensively on the Adafruit feather nrf52840 express, but not on any other boards. With the conditional compilation around most of the new DFU activation pin functionality and QSPI flash, regressions to other boards are not expected, although testing these would be prudent.

…ctivation takes precedence. (Some solutions don't need FRST for BLE OTA updates, and those that do can define FRST as a separate pin)

…er being active when the DFU Activation pin is asserted

…ity.

…e cleanup.

# Conflicts: # src/main.c

bsatrom · 2022-11-08T15:20:16Z

@hathach can you take a look at this PR, let us know if the changes are acceptable or if not, what we need to do to get it into a good spot for merging into main?

hathach · 2022-11-18T10:45:27Z

sorry, I was a bit busy for now, will check this out whenever I could !!

bsatrom · 2022-12-09T20:59:38Z

Hey @hathach have you had a chance to review this yet?

m-mcgowan · 2023-02-14T20:17:30Z

@hathach Is there any way I can help you with reviewing this? I'm happy to walk you though the PR and talk through the changes, the motivation behind them etc. It would be great to have this merged, so anything I can do to help just let me know. Thanks!

hathach · 2023-03-01T08:54:48Z

sorry for late response, I am in the middle of reviewing this, however look like the additional code cause the flash to be overflowed. Let me think how to shrink the size of bootloader quickly first, then we can merge to this and continue with the PR.

LD feather_nrf52840_express_bootloader-0.7.0-16-g047531a-dirty.out
/home/hathach/.local/xPacks/@xpack-dev-tools/arm-none-eabi-gcc/11.2.1-1.2.2/.content/bin/../lib/gcc/arm-none-eabi/11.2.1/../../../../arm-none-eabi/bin/ld: _build/build-feather_nrf52840_express/feather_nrf52840_express_bootloader-0.7.0-16-g047531a-dirty.out section `.text' will not fit in region `FLASH'
/home/hathach/.local/xPacks/@xpack-dev-tools/arm-none-eabi-gcc/11.2.1-1.2.2/.content/bin/../lib/gcc/arm-none-eabi/11.2.1/../../../../arm-none-eabi/bin/ld: region FLASH overflowed with .data and user data
/home/hathach/.local/xPacks/@xpack-dev-tools/arm-none-eabi-gcc/11.2.1-1.2.2/.content/bin/../lib/gcc/arm-none-eabi/11.2.1/../../../../arm-none-eabi/bin/ld: section .bootloaderConfig LMA [00000000000fd800,00000000000fd857] overlaps section .text LMA [00000000000f4000,00000000000fd9e7]
/home/hathach/.local/xPacks/@xpack-dev-tools/arm-none-eabi-gcc/11.2.1-1.2.2/.content/bin/../lib/gcc/arm-none-eabi/11.2.1/../../../../arm-none-eabi/bin/ld: region `FLASH' overflowed by 728 bytes
collect2: error: ld returned 1 exit status

PS: oops, never mind, it is due to my setup with debug option. All is good now, but I am making an PR to optimize space anyway. Will continue with reviewing

hathach · 2023-03-01T16:03:27Z

src/main.c

-    bootloader_app_start();
+    // register the app
+    uint32_t app_addr = bootloader_app_register();
+    if (!dfu_activate) {


since we reset when dfu_activate=1, we can just skip the teardown and simply reset. Teardown is only needed when jumping to app. I will make changes accordingly.

hathach · 2023-03-01T16:50:21Z

I need to push changes to your fork to update PR, would you mind give me the write permission ?

m-mcgowan · 2023-03-01T19:11:50Z

hathach

Sure thing, this is done! Thank you for starting to review.

hathach · 2023-03-02T03:07:30Z

src/main.c

+  // binaries to flash (e.g. QSPI flash data)
+  if (PIN_DFU_ACTIVATE_PRESENT && dfu_activate)
+  {
+    NVIC_SystemReset();


reset immediately here with activation would reduce the modification of existing code. I don't have the set up to test, please double check to see if this commit stills work on your side. Then we can move on with the reviewing (it may take a couple more commits I think).

I understand the tear down isn't strictly necessary, but it does no harm either, right?!. :-) I would vote for leaving the code as is, since it's already well tested. I only changed the code that was strictly necessary to keep the changes to the bare minimum.

If you feel very strongly for the change then of course we can test it.

please do the testing if possible, reset here will allow us to keep the rest of main.c unchanged, we could also revert changes to the bootloader.c file as well. I will also make a couple more commits anyway mostly to moving/refactoring changes.

I will test this today and will post back with results. Thanks!

Unfortunately it didn't work. I can try to find out why later, but from the perspective of the programmer, everything was ok. All the packets were ACKed and it successfully programmed the CPy interpreter and an image for the flash filesystem. But on reset, the nRF stays in the bootloader.

I'll look deeper into it and check the memory contents and look for other clues as to why it's not working.

hmm, that is weird, probably one of the teardown function is indeed required. I will also try to do some check as well

I'll review the changes today to see if I have any suggestions.

I reviewed the changes and retested for a few hours. Initially things weren't working for me, but eventually I managed to make it work with no code changes needed. I'm going to retest today (and write up the testing process since it's quite involved and easy to make a mistake.)

I'll let you know when the testing is done and we can merge this soon!

I'm still seeing intermittent errors. It is like the application doesn't "stick".

I don't have the root cause, but we are able to make this work in cases when the application is the last binary flashed (so qspi filesystem then the app). When the application is flashed first, then the QSPI filesystem, the application isn't saved. I believe the error is on our end (most likely related to timing), because I see the same result when I revert the recent changes to the PR back to my original PR, which I know was working, so something else must have changed. So I would say this PR is good to merge as is.

GlibSkunk · 2024-05-08T00:47:12Z

I would also appreciate the DFU over UART pins for the Feather nRF52840 Express, so thanks for the work guys! One request though if possible... could a new "Magic Number" also be added to the GPREGRET which points directly to DFU over UART?

In my scenario I communicate with the feather through UART exclusively. In my application code, I would then put in a uart command to set GPREGRET to the "DFU_MAGIC_SERIAL_UART_ONLY_RESET" and trigger a reset. This would avoid requiring an additional pull on the DFU pin.

First time Github poster, so I hope this request isn't completely out of line.

m-mcgowan added 8 commits June 23, 2022 09:54

adds a pin for DFU activation. If this is the same pin as FRST, dfu a…

83490cc

…ctivation takes precedence. (Some solutions don't need FRST for BLE OTA updates, and those that do can define FRST as a separate pin)

makes both USB CDC and UART comms available at runtime, with the latt…

3a047c0

…er being active when the DFU Activation pin is asserted

feat: adds support for programming the external qspi flash

2954894

ensure the QSPI flash is initialized. Adjust the QSPI interrupt prior…

0d29fa0

…ity.

revert lib/nrfx to 2.0.0 - causes compilation errors on nRF52832. cod…

0c2fe0d

…e cleanup.

remove unneeded changes to the LED signalling

7b78218

Merge branch 'master' into dfu_activation_pin

62b53ae

# Conflicts: # src/main.c

fix: typo in function name

047531a

m-mcgowan changed the title ~~nRF52840: MCU to MCU updates via activation pin and QSPI~~ nRF52840: MCU to MCU updates via activation pin and CircuitPython filesystem programming Oct 21, 2022

hathach reviewed Mar 1, 2023

View reviewed changes

make some changes for quick reset when dfu activation

256e174

hathach reviewed Mar 2, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nRF52840: MCU to MCU updates via activation pin and CircuitPython filesystem programming #283

nRF52840: MCU to MCU updates via activation pin and CircuitPython filesystem programming #283

m-mcgowan commented Oct 21, 2022 •

edited

Loading

bsatrom commented Nov 8, 2022

hathach commented Nov 18, 2022

bsatrom commented Dec 9, 2022

m-mcgowan commented Feb 14, 2023

hathach commented Mar 1, 2023 •

edited

Loading

hathach Mar 1, 2023

hathach commented Mar 1, 2023

m-mcgowan commented Mar 1, 2023

hathach Mar 2, 2023

m-mcgowan Mar 2, 2023

hathach Mar 2, 2023 •

edited

Loading

m-mcgowan Mar 3, 2023

m-mcgowan Mar 4, 2023

hathach Mar 6, 2023

m-mcgowan Mar 7, 2023 •

edited

Loading

m-mcgowan Mar 16, 2023

m-mcgowan Mar 17, 2023

m-mcgowan Mar 22, 2023

GlibSkunk commented May 8, 2024

nRF52840: MCU to MCU updates via activation pin and CircuitPython filesystem programming #283

Are you sure you want to change the base?

nRF52840: MCU to MCU updates via activation pin and CircuitPython filesystem programming #283

Conversation

m-mcgowan commented Oct 21, 2022 • edited Loading

Overview

Configuration Changes

Other Implementation Details

Testing

bsatrom commented Nov 8, 2022

hathach commented Nov 18, 2022

bsatrom commented Dec 9, 2022

m-mcgowan commented Feb 14, 2023

hathach commented Mar 1, 2023 • edited Loading

Choose a reason for hiding this comment

hathach commented Mar 1, 2023

m-mcgowan commented Mar 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hathach Mar 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m-mcgowan Mar 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GlibSkunk commented May 8, 2024

m-mcgowan commented Oct 21, 2022 •

edited

Loading

hathach commented Mar 1, 2023 •

edited

Loading

hathach Mar 2, 2023 •

edited

Loading

m-mcgowan Mar 7, 2023 •

edited

Loading