Skip to content

Commit

Permalink
build: Use workspace to group rust sub-projects
Browse files Browse the repository at this point in the history
meson build script was building each rust sub-project under rust/ and
scheds/rust/ separately. This means that each rust project is built
independently which leads to a couple problems - 1. There are a lot of
shared dependencies but they have to be built over and over again for each
proejct. 2. Concurrency management becomes sad - we either have to unleash
multiple cargo builds at the same time possibly thrashing the system or
build one by one.

We've been trying to solve this from meson side in vain. Thankfully, in
issue sched-ext#546, @vimproved suggested using cargo workspace which makes the
sub-projects share the same target directory and built together by the same
cargo instance while still allowing each project to behave independently for
development and publishing purposes.

Make the following changes:

- Create two cargo workspaces - one under rust/, the other under
  scheds/rust/. Each contains all rust projects underneath it.

- Don't let meson descend into rust/. These are libraries used by the rust
  schedulers. No need to build them from meson. Cargo will build them as
  needed.

- Change the rust_scheds build target to invoke `cargo build` in
  scheds/rust/ and let cargo do its thing.

- Remove per-scheduler meson.build files and instead generate custom_targets
  in scheds/rust/meson.build which invokes `cargo build -p $SCHED`.

- This changes rust binary directory. Update README and
  meson-scripts/install_rust_user_scheds accordingly.

- Remove per-scheduler Cargo.lock as scheds/rust/Cargo.lock is shared by all
  schedulers now.

- Unify .gitignore handling.

The followings are build times on Ryzen 3975W:

Before:
  ________________________________________________________
  Executed in  165.93 secs    fish           external
     usr time   40.55 mins    2.71 millis   40.55 mins
     sys time    3.34 mins   36.40 millis    3.34 mins

After:
  ________________________________________________________
  Executed in   36.04 secs    fish           external
     usr time  336.42 secs    0.00 millis  336.42 secs
     sys time   36.65 secs   43.95 millis   36.61 secs

Wallclock time is reduced 5x and CPU time 7x.
  • Loading branch information
htejun committed Aug 25, 2024
1 parent c93191a commit 43950c6
Show file tree
Hide file tree
Showing 36 changed files with 191 additions and 9,969 deletions.
5 changes: 1 addition & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,9 +244,7 @@ For example, here, the `scx_simple` binary can be found at
`$SCX/build/scheds/c/scx_simple`.

For Rust userspace schedulers such as the ones under `scheds/rust`, the
same directory under the build root is used as the cargo build target
directory. Thus, here, the `scx_rusty` binary can be found at
`$SCX/build/scheds/rust/scx_rusty/release/scx_rusty`.
`scx_rusty` binary can be found at `$SCX/build/scheds/rust/release`.


### SCX specific build options
Expand All @@ -264,7 +262,6 @@ options can be used in such cases.
- 'cargo_home': 'CARGO_HOME env to use when invoking cargo'
- `offline`: 'Compilation step should not access the internet'
- `enable_rust`: 'Enable the build of rust sub-projects'
- `serialize`: 'Enable/disable the sequential build of the schedulers. Set this to false if you need to build just one scheduler.'

For example, let's say you want to use `bpftool` and `libbpf` shipped in the
kernel tree located at `$KERNEL`. We need to build `bpftool` in the kernel
Expand Down
4 changes: 2 additions & 2 deletions meson-scripts/install_rust_user_scheds
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ set -e

for manifest in "$MESON_SOURCE_ROOT"/scheds/rust/*/Cargo.toml; do
source_dir="${manifest%/Cargo.toml}"
target_dir="${MESON_BUILD_ROOT}${source_dir#${MESON_SOURCE_ROOT}}"
name="${target_dir##*/}"
target_dir="${MESON_BUILD_ROOT}/scheds/rust"
name="${source_dir##*/}"

# Skip scx_mitosis
if [ "$name" = "scx_mitosis" ]; then
Expand Down
5 changes: 0 additions & 5 deletions meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,6 @@ cc = meson.get_compiler('c')

enable_rust = get_option('enable_rust')

serialize = get_option('serialize')

bpf_clang = find_program(get_option('bpf_clang'))

run_veristat = find_program(join_paths(meson.current_source_dir(),
Expand Down Expand Up @@ -338,9 +336,6 @@ run_target('veristat_diff', command: [run_veristat_diff, meson.current_build_dir
get_option('veristat_scheduler'), get_option('kernel'),
get_option('veristat_diff_dir')])

if enable_rust
subdir('rust')
endif
subdir('scheds')

if enable_stress
Expand Down
2 changes: 0 additions & 2 deletions meson.options
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ option('offline', type: 'boolean', value: 'false',
description: 'Compilation step should not access the internet')
option('enable_rust', type: 'boolean', value: 'true',
description: 'Enable rust sub-projects')
option('serialize', type: 'boolean', value: 'true',
description: 'Serialize the build of all schedulers')
option('enable_stress', type: 'boolean', value: 'true',
description: 'Enable stress tests')
option('kernel', type: 'string', value: 'vmlinuz',
Expand Down
File renamed without changes.
3 changes: 3 additions & 0 deletions rust/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[workspace]
members = ["scx_utils", "scx_stats", "scx_stats/scx_stats_derive", "scx_rustland_core"]
resolver = "2"
3 changes: 0 additions & 3 deletions rust/meson.build

This file was deleted.

1 change: 0 additions & 1 deletion rust/scx_rustland_core/.gitignore

This file was deleted.

8 changes: 0 additions & 8 deletions rust/scx_rustland_core/meson.build

This file was deleted.

2 changes: 0 additions & 2 deletions rust/scx_stats/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
workspace = { members = ["scx_stats_derive"] }

[package]
name = "scx_stats"
version = "1.0.3"
Expand Down
7 changes: 0 additions & 7 deletions rust/scx_stats/meson.build

This file was deleted.

2 changes: 0 additions & 2 deletions rust/scx_utils/.gitignore

This file was deleted.

8 changes: 0 additions & 8 deletions rust/scx_utils/meson.build

This file was deleted.

2 changes: 2 additions & 0 deletions scheds/rust/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*/src/bpf/.output
target
166 changes: 146 additions & 20 deletions scheds/rust/scx_rusty/Cargo.lock → scheds/rust/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions scheds/rust/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[workspace]
members = ["scx_lavd", "scx_bpfland", "scx_rustland", "scx_rlfifo",
"scx_rusty",
"scx_layered"] # scx_mitosis temporarily excluded
resolver = "2"
Loading

0 comments on commit 43950c6

Please sign in to comment.