Dynamic Userland Application Loading #3941

viswajith-g · 2024-03-28T22:18:28Z

Pull Request Overview

This pull request adds dynamic userland application loading to the kernel.

There are three stages to this:

Setup Phase
Flash Phase
Load Phase

Setup Phase

During the setup phase, a userland application passes the size of the new binary to the app_loader capsule.
This capsule forwards the size to the kernel which determines if there is enough space in the flash to write the new app and what address the app should be written to. On success, the kernel returns the application size it has set up for to the capsule.
The capsule returns a success to the userland app.
On Failure, the kernel passes the reason for failure to the capsule, and the capsule passes a FAIL error to the userland app.

Flash Phase

After the userland app receives the Ok from the capsule, the app sends the binary of the new app 512 bytes at a time along with the offset.
The capsule checks that these values do not violate the bounds dictated by the kernel and then passes the binary data to the kernel.
Upon receiving the binary data, the kernel once again checks if the data is within the bounds specified during the setup phase and then writes the data to flash. On success, the kernel passes success to the app via the capsule. On failure, the same is propagated to the userland application.

Load Phase

Once the userland app receives confirmation that the final set of writes is completed, the app sends a load request to the kernel via the capsule. The kernel looks for the process binary at the address we just finished writing and converts the app binary into a process object. Upon success, the kernel tries to add the process object to the processes array. Once this is done, the process is officially active, and the kernel writes padding after the newly installed app to preserve the continuation of the linked list for future app loading.

Testing Strategy

This pull request was tested by running the app_loader application, a test userland app to verify if the new app (blink) was written and loaded correctly on nRF52840DK. This was tested in three configurations:

When only the app_loader user app is installed on the device.
- In this case, the dynamic app loader writes the new app after the app_loader user app and then loads it.
When there is another app in addition to app_loader.
- The device had two apps installed on it, one was adc and the other was app_loader. In this case, the dynamic process loader finds available flash after the two apps and writes blink there after which it loads the new app.
When there is padding in between applications.
- In this case, the device had padding, then app_loader and more padding. Dynamic process loader fits the app in the padding area before app_loader, and writes new padding between blink and app_loader.

A previous implementation of the process loader without the process binaries is available at this repo. This version was tested on the Imix board, and given not too much has changed, I think the new implementation should work on Imix.

TODO or Help Wanted

More testing.
Feedback and merge.

Documentation Updated

✅ Updated the relevant files in /docs, or no updates are required.

Formatting

✅ Ran make prepush.

This version works by writing the app first and then creates the process and adds it to the processes array using the synchronous process loading methods. Finally it writes the padding app. There are new overheads introduced however in the form of load_processes_return_memory() and load_processes_from_flash_return() in process_loading.rs. This is because, when the board initially does setup, we want it to pass the remaining RAM available to the dynamic process loader to load new apps. Currently load_processes() does not return the remaining memory, so those two new methods were added.

fixed some warnings causing CI fail

kernel/src/dynamic_process_loading.rs

changed it so that the capsule sends a subslice of the buffer to the kernel so the kernel does not have to compare lengths that the capsule provided vs the length of the buffer the capsule actually sent. Generally a safer approach.

Improved the state machine to better track if a userland app is requesting write to the bounds allocated to it during the setup phase. This state machine also ensures that when the process loader is busy, another request cannot be made.

Improved the state machine to better track if a userland app is requesting write to the bounds allocated to it during the setup phase. Also tracks if the dynamic process loader is currently busy. Performed some code cleanup.

The kernel now validates the header before writing the app into flash. In addition, upon failure at any stage, the flash region will be reclaimed for future use. Added a Fail state to help track failure modes better.

fixed two instances in app write when resets were not taking place. also fixed an improper condition check for declared length vs length in header.

An app that was successfully could not be made into a process object because of errant state tracking. That was fixed. Additionally, removed unnecessary state and parameter clears during Busy state.

kernel/src/process_loading.rs

kernel/src/dynamic_process_loading.rs

kernel/src/process_loading.rs

Apps were previously aligned but we were not writing padding before new apps to match the linkedlist with the alignment. This is now fixed. A new enum was added to track the padding requirement. Additionally, the setup phase is now asynchronous to accomodate cases where you may or may not have to write padding after an app depending on if there is an app stored beyond our newly written app. This change propogates to the capsule as well.

check_padding_requirements was called twice once during postpad and during prepad to return the next_app_start_addr and previous_app_end_addr respectively. Changed the function so that the values are stored and the function does not have to be called twice.

changed the way the app address is identified during setup phase to align the app to the power of 2 based on its size and start address.

capsules/extra/src/app_loader.rs

kernel/src/dynamic_process_loading.rs

Moved some logic to improve flow. Changed write so that we don't track write validity based on offset increments. Added checks to make sure the header is not manipulated without changing the whole header.

capsules/extra/src/app_loader.rs

kernel/src/dynamic_process_loading.rs

bradjc · 2024-05-09T21:56:49Z

kernel/src/dynamic_process_loading.rs

+            match self.find_next_available_address(flash, app_length) {
+                Ok(new_app_start_address) => {
+                    let offset = new_app_start_address - flash_start;
+                    let new_process_flash = self
+                        .flash
+                        .get()
+                        .get(offset..offset + app_length)
+                        .ok_or(ErrorCode::FAIL)?;
+
+                    self.new_process_flash.set(new_process_flash);
+                    self.new_app_start_addr.set(new_app_start_address);
+                    self.new_app_length.set(app_length);
+
+                    match self.padding_requirement.get() {


This is too confusing. find_next_available_address() should just do a find and return everything (including the need for padding). That state is mutated deep in here makes it hard to reason about how this works.

kernel/src/dynamic_process_loading.rs

bradjc · 2024-06-24T13:41:14Z

kernel/src/dynamic_process_metadata.rs

Remove file.

kernel/src/process_loading.rs

Removed kernel warnings

kernel/src/process_loading.rs

boards/configurations/nrf52840dk/nrf52840dk-test-dynamic-app-load/src/main.rs

viswajith-g · 2024-07-04T01:50:42Z

Ok, I don't know why it keeps saying the documentation for app_loader.rs is missing.

bradjc · 2024-07-04T01:58:18Z

Can you remove kernel/src/dynamic_process_metadata.rs?

viswajith-g · 2024-07-04T02:22:45Z

Can you remove kernel/src/dynamic_process_metadata.rs?

If I do that, make prepush fails, but the code works as intended. Is that an issue?

vis@Vis:~/rebase_apploader/tock$ make prepush
make: *** No rule to make target 'kernel/src/dynamic_process_metadata.rs', needed by 'tools/.format_fresh'.  Stop.

bradjc · 2024-07-04T04:27:31Z

No.

And as a side note, that will be fixed when #4037 is merged.

deleted kernel/src/dynamic_process_metadata.rs because it was an empty file.

Co-authored-by: Brad Campbell <bradjc5@gmail.com>

bradjc · 2024-07-05T21:00:19Z

Ok great I think this is to a point where I can take a deeper look and actually try it. I will try to do that soon.

bradjc · 2024-07-10T17:23:55Z

kernel/src/dynamic_process_loading.rs

+    fn check_overlap_region(
+        &self,
+        new_start_address: usize,
+        app_length: usize,
+    ) -> Result<(), (usize, ProcessLoadError)> {
+        // Find the next open process slot.
+        let new_process_count = self.find_open_process_slot().unwrap_or_default();
+        let new_process_start_address = new_start_address;
+        let new_process_end_address = new_process_start_address + app_length - 1;
+
+        self.procs.map(|procs| {
+            for (proc_index, value) in procs.iter().enumerate() {
+                if proc_index < new_process_count {
+                    let process_start_address = value.unwrap().get_addresses().flash_start;
+                    let process_end_address = value.unwrap().get_addresses().flash_end;


Unfortunately this isn't going to work. The main issue is that there is no requirement that loaded processes (ie in the PROCESSES array) represent all stored process binaries.

What we probably need to do instead is use the ProcessBinaries array.

bradjc · 2024-07-10T17:46:07Z

I've gone through most of the main kernel library, cleaning things up and making the comments consistent with the kernel crate.

I removed the unsafe, and I think this PR will build again with #4079.

I was hoping to be able to test this but we need to improve the logic around searching for a window to store the new process into. Also, we shouldn't use unwrap(), and in this case the current uses are red flags that the code isn't quite right.

alevy · 2024-08-30T16:27:05Z

Work expected to resume fall semester '24

bradjc · 2025-01-17T22:39:07Z

Todo:

Implement the logic for finding a suitable region in flash within the sequential process loader.
Move the new dynamic process storage mechanism to its own file (separate from the trait).

previous iteration erroneously used the size of the tbf header instead of the size of an already present application to compute the available address for a new app

bradjc · 2025-01-31T23:02:10Z

capsules/core/src/driver.rs

@@ -57,6 +57,7 @@ pub enum NUM {
    NvmStorage            = 0x50001,
    SdCard                = 0x50002,
    Kv                    = 0x50003,
+    AppLoader             = 0x50004,


This should be 0x10001 I think,

Moved the binary header validity check to process_loading, eliminating the need to track process flash slice in DPL.

viswajith-g added 5 commits March 28, 2024 12:44

fixed warnings

a331d70

fixed some warnings causing CI fail

fixed some format issues

e4073f1

clippy fixes

b575bec

fixed header check issue

12d6e60

github-actions bot added kernel component labels Mar 28, 2024

bradjc reviewed Mar 28, 2024

View reviewed changes

resolved some pr comments

aa51671

bradjc reviewed Mar 29, 2024

View reviewed changes

viswajith-g added 3 commits March 31, 2024 19:27

pr comments changes

2b54b13

changed it so that the capsule sends a subslice of the buffer to the kernel so the kernel does not have to compare lengths that the capsule provided vs the length of the buffer the capsule actually sent. Generally a safer approach.

Improved State Machine

8181efc

Improved the state machine to better track if a userland app is requesting write to the bounds allocated to it during the setup phase. This state machine also ensures that when the process loader is busy, another request cannot be made.

Improved State Machine

32b288b

Improved the state machine to better track if a userland app is requesting write to the bounds allocated to it during the setup phase. Also tracks if the dynamic process loader is currently busy. Performed some code cleanup.

viswajith-g mentioned this pull request Apr 1, 2024

Dynamic App Loader Helper tock/libtock-c#387

Open

viswajith-g added 3 commits April 1, 2024 19:03

Header validation + Erasing app upon failure

05f4207

The kernel now validates the header before writing the app into flash. In addition, upon failure at any stage, the flash region will be reclaimed for future use. Added a Fail state to help track failure modes better.

bug fixes

a2156ed

fixed two instances in app write when resets were not taking place. also fixed an improper condition check for declared length vs length in header.

states bug fix

8c9e836

An app that was successfully could not be made into a process object because of errant state tracking. That was fixed. Additionally, removed unnecessary state and parameter clears during Busy state.

viswajith-g requested a review from bradjc April 2, 2024 02:40

alistair23 reviewed Apr 2, 2024

View reviewed changes

kernel/src/process_loading.rs Outdated Show resolved Hide resolved

alistair23 reviewed Apr 2, 2024

View reviewed changes

kernel/src/dynamic_process_loading.rs Outdated Show resolved Hide resolved

alistair23 reviewed Apr 2, 2024

View reviewed changes

kernel/src/process_loading.rs Outdated Show resolved Hide resolved

github-actions bot assigned bradjc Apr 6, 2024

viswajith-g added 3 commits April 5, 2024 22:28

alignment improvement

e67b478

changed the way the app address is identified during setup phase to align the app to the power of 2 based on its size and start address.

bradjc reviewed Apr 11, 2024

View reviewed changes

Better flow and documentation

c7fe16b

Moved some logic to improve flow. Changed write so that we don't track write validity based on offset increments. Added checks to make sure the header is not manipulated without changing the whole header.

bradjc reviewed May 9, 2024

View reviewed changes

alevy added needs-rebase waiting-on-author labels Jun 6, 2024

bradjc removed needs-rebase waiting-on-author labels Jun 28, 2024

bradjc reviewed Jun 28, 2024

View reviewed changes

Clean up

bcc1f2e

Removed kernel warnings

bradjc reviewed Jul 4, 2024

View reviewed changes

kernel/src/process_loading.rs Outdated Show resolved Hide resolved

boards/configurations/nrf52840dk/nrf52840dk-test-dynamic-app-load/src/main.rs Outdated Show resolved Hide resolved

viswajith-g added 2 commits July 3, 2024 21:39

additional cleanup

8e148fd

address CI failure

204b582

viswajith-g and others added 2 commits July 4, 2024 00:38

removed unneeded file

ccccc3d

deleted kernel/src/dynamic_process_metadata.rs because it was an empty file.

Better explain how write works

e03e84d

Co-authored-by: Brad Campbell <bradjc5@gmail.com>

bradjc added 2 commits July 10, 2024 00:02

kernel: dpl: comments and simplify

ad6ae27

kernel: dpl: simplify find_next_available_address

ad85b09

bradjc reviewed Jul 10, 2024

View reviewed changes

alevy added the waiting-on-author label Aug 30, 2024

ppannuto mentioned this pull request Nov 18, 2024

[Feature request] Support for dynamic process loading/in-place execution #4243

Open

viswajith-g and others added 2 commits January 28, 2025 19:09

fix new app address compute

3421809

previous iteration erroneously used the size of the tbf header instead of the size of an already present application to compute the available address for a new app

moved setup minus writing padding to process_loading.rs

797f3e2

bradjc reviewed Jan 31, 2025

View reviewed changes

github-actions bot added the HIL This affects a Tock HIL interface. label Feb 1, 2025

viswajith-g force-pushed the master branch from 540c95f to 797f3e2 Compare February 1, 2025 02:19

github-actions bot removed the HIL This affects a Tock HIL interface. label Feb 1, 2025

viswajith-g and others added 2 commits January 31, 2025 21:32

Merge branch 'master' of https://github.com/tock/tock

968ab04

Moved more flash access from DPL to ProcessLoadingAsync

99423b0

Moved the binary header validity check to process_loading, eliminating the need to track process flash slice in DPL.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dynamic Userland Application Loading #3941

Dynamic Userland Application Loading #3941

viswajith-g commented Mar 28, 2024 •

edited

Loading

bradjc May 9, 2024

bradjc Jun 24, 2024

viswajith-g commented Jul 4, 2024

bradjc commented Jul 4, 2024

viswajith-g commented Jul 4, 2024

bradjc commented Jul 4, 2024

bradjc commented Jul 5, 2024

bradjc Jul 10, 2024

bradjc commented Jul 10, 2024

alevy commented Aug 30, 2024

bradjc commented Jan 17, 2025

bradjc Jan 31, 2025

Dynamic Userland Application Loading #3941

Are you sure you want to change the base?

Dynamic Userland Application Loading #3941

Conversation

viswajith-g commented Mar 28, 2024 • edited Loading

Pull Request Overview

Setup Phase

Flash Phase

Load Phase

Testing Strategy

TODO or Help Wanted

Documentation Updated

Formatting

bradjc May 9, 2024

Choose a reason for hiding this comment

bradjc Jun 24, 2024

Choose a reason for hiding this comment

viswajith-g commented Jul 4, 2024

bradjc commented Jul 4, 2024

viswajith-g commented Jul 4, 2024

bradjc commented Jul 4, 2024

bradjc commented Jul 5, 2024

bradjc Jul 10, 2024

Choose a reason for hiding this comment

bradjc commented Jul 10, 2024

alevy commented Aug 30, 2024

bradjc commented Jan 17, 2025

bradjc Jan 31, 2025

Choose a reason for hiding this comment

viswajith-g commented Mar 28, 2024 •

edited

Loading