Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: kernel virtmem cont #189

Merged
merged 42 commits into from
Dec 29, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
208e3b0
fix(wavltree): return tree lifetimes from `Cursor`
JonasKruckenberg Dec 18, 2024
c1e2e78
feat(loader): report stacks & TLS region and online hart mask
JonasKruckenberg Dec 18, 2024
f3baa84
wip
JonasKruckenberg Dec 18, 2024
00d08e2
Merge branch 'main' into jonas/feat/kernel-virtmem
JonasKruckenberg Dec 18, 2024
0f7372e
fmt & clippy
JonasKruckenberg Dec 18, 2024
2b5a77b
Update aspace.rs
JonasKruckenberg Dec 18, 2024
e1685b6
refactor: properly setup address space range constants
JonasKruckenberg Dec 19, 2024
ac64c0f
fix tests
JonasKruckenberg Dec 19, 2024
a254b3f
fmt
JonasKruckenberg Dec 19, 2024
e83c1ca
fix: fix tests by identity mapping RTC driver
JonasKruckenberg Dec 19, 2024
5d4c116
fmt
JonasKruckenberg Dec 19, 2024
5fb8160
wip
JonasKruckenberg Dec 19, 2024
1768a22
refactor: rename `pmm` to `mmu`
JonasKruckenberg Dec 19, 2024
04c2a8f
refactor(loader): report physical address offset and physical memory …
JonasKruckenberg Dec 22, 2024
d7b3564
fix(mmu): report error if Flush is dropped with unflushed changes
JonasKruckenberg Dec 22, 2024
c299420
wip
JonasKruckenberg Dec 22, 2024
e6417ea
refactor(mmu): allow any `FramesIterator` implementation
JonasKruckenberg Dec 22, 2024
f7d4abc
fix(dtb-parser): implement `Iterator` for `Strings`
JonasKruckenberg Dec 22, 2024
7fcd97b
refactor: don't explicitly map RTC device
JonasKruckenberg Dec 22, 2024
f7d2deb
chore(mmu): better printing of addresses
JonasKruckenberg Dec 22, 2024
55cbfe1
fix(wavltree): return `Pin<&mut T>` from `insert`
JonasKruckenberg Dec 22, 2024
d91e4da
fix(mmu): correct pretty printing of addresses
JonasKruckenberg Dec 25, 2024
ab95d6e
wip
JonasKruckenberg Dec 25, 2024
dfe403b
fmt & fixes
JonasKruckenberg Dec 26, 2024
5c2feec
fix: expand initial heap for elf processing
JonasKruckenberg Dec 26, 2024
d3ca827
fix startup
JonasKruckenberg Dec 26, 2024
a21a111
fix: print stack trace on panic
JonasKruckenberg Dec 26, 2024
e4c6580
feat: support short backtrace
JonasKruckenberg Dec 26, 2024
119d3bf
fix tests
JonasKruckenberg Dec 26, 2024
240e2af
don't panic in drop
JonasKruckenberg Dec 26, 2024
86a216b
fix(kernel/vm): don't panic when virt allocation range is empty
JonasKruckenberg Dec 26, 2024
e8e2ece
fmt
JonasKruckenberg Dec 26, 2024
188ea02
fix(loader): prevent frame allocator from handing out loader memory
JonasKruckenberg Dec 28, 2024
b66b360
test: add tests for `mmu` crate
JonasKruckenberg Dec 28, 2024
22d9609
fix: better address arithmetic
JonasKruckenberg Dec 28, 2024
ca9f574
fix(mmu): correct `align_down` implementation
JonasKruckenberg Dec 28, 2024
2f999ab
tests(mmu): fix tests
JonasKruckenberg Dec 28, 2024
13c85c3
fix(kernel/vm): return gap size zero instead of panic
JonasKruckenberg Dec 28, 2024
485e14b
Merge branch 'main' into jonas/feat/kernel-virtmem
JonasKruckenberg Dec 29, 2024
f53a165
enable debug assertions in release mode
JonasKruckenberg Dec 29, 2024
3d13f23
fix: increase QEMU physical memory for tests
JonasKruckenberg Dec 29, 2024
3a0c09d
clippy & fmt
JonasKruckenberg Dec 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fmt & fixes
  • Loading branch information
JonasKruckenberg committed Dec 26, 2024
commit dfe403bf36c1a7a3a4d0a1983f36b6f7447494db
14 changes: 3 additions & 11 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,13 @@ arrayvec = { version = "0.7.4", default-features = false }
object = { version = "0.36.4", default-features = false }
gimli = { version = "0.31.0", default-features = false, features = ["read"] }
xmas-elf = "0.9.1"
talc = "4.4.2"
talc = { version = "4.4.2", default-features = false }
pin-project-lite = "0.2.15"
rand_chacha = { version = "0.3.1", default-features = false }
rand = { version = "0.8.5", default-features = false }
static_assertions = "1.1.0"
addr2line = { version = "0.24.2", default-features = false }
rustc-demangle = { version = "0.1.24", default-features = false }

# wast dependencies
bumpalo = "3.14.0"
Expand Down
4 changes: 2 additions & 2 deletions kernel/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ harness = false
loader-api.workspace = true
mmu.workspace = true
panic-unwind.workspace = true
semihosting-logger = { workspace = true, features = ["hartid"] }
sync.workspace = true
thread-local.workspace = true
wavltree= { workspace = true, features = ["dot"] }
dtb-parser.workspace = true
backtrace.workspace = true

talc.workspace = true
talc = { workspace = true, features = ["lock_api", "counters"] }
log.workspace = true
cfg-if.workspace = true
pin-project-lite.workspace = true
Expand Down
10 changes: 8 additions & 2 deletions kernel/src/arch/riscv64/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,17 @@ pub fn wait_for_interrupt() {
unsafe { asm!("wfi") }
}

/// Finish architecture-specific, hart-local initialization.
pub fn finish_hart_init() {
/// Early architecture-specific, per-hart initialization.
pub fn hart_init_early() {}

/// Late architecture-specific, per-hart initialization.
pub fn hart_init_late() {
unsafe {
// Enable interrupts
interrupt::enable();
// Enable supervisor timer interrupts
sie::set_stie();
// Enable FPU
sstatus::set_fs(FS::Initial);
}
}
7 changes: 2 additions & 5 deletions kernel/src/arch/riscv64/vm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,14 @@ use mmu::{AddressRangeExt, Flush, PhysicalAddress, VirtualAddress};

const KERNEL_ASID: usize = 0;

pub fn init(
boot_info: &BootInfo,
minfo: &MachineInfo,
) -> crate::Result<mmu::AddressSpace> {
pub fn init(boot_info: &BootInfo, minfo: &MachineInfo) -> crate::Result<mmu::AddressSpace> {
let mut frame_alloc = crate::vm::FRAME_ALLOC.get().unwrap().lock();

let (mut arch, mut flush) =
mmu::AddressSpace::from_active(KERNEL_ASID, boot_info.physical_address_offset);

unmap_loader(boot_info, &mut arch, &mut flush);

flush.flush()?;

Ok(arch)
Expand Down
1 change: 1 addition & 0 deletions kernel/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ extern crate alloc;
mod allocator;
pub mod arch;
mod error;
mod logger;
pub mod machine_info;
mod start;
pub mod time;
Expand Down
51 changes: 51 additions & 0 deletions kernel/src/logger.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
use core::cell::RefCell;
use core::fmt::Write;
use log::{LevelFilter, Metadata, Record};
use thread_local::declare_thread_local;

/// Initializes the global logger with the semihosting logger.
///
/// # Panics
///
/// This function will panic if it is called more than once, or if another library has already initialized a global logger.
pub fn init(lvl: LevelFilter) {
static LOGGER: Logger = Logger;

log::set_logger(&LOGGER).unwrap();
log::set_max_level(lvl);
}

pub fn init_hart(hartid: usize) {
let hio = riscv::hio::HostStream::new_stdout();

STDOUT.initialize_with((RefCell::new(hio), hartid), |_, _| {});
}

declare_thread_local!(
static STDOUT: (RefCell<riscv::hio::HostStream>, usize);
);

struct Logger;

impl log::Log for Logger {
fn enabled(&self, _metadata: &Metadata) -> bool {
true
}

fn log(&self, record: &Record) {
if self.enabled(record.metadata()) {
STDOUT.with(|(stdout, hartid)| {
let mut stdout = stdout.borrow_mut();
let _ = stdout.write_fmt(format_args!(
"[{:<5} HART {} {}] {}\n",
record.level(),
hartid,
record.module_path_static().unwrap_or_default(),
record.args()
));
});
}
}

fn flush(&self) {}
}
6 changes: 3 additions & 3 deletions kernel/src/machine_info.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ impl fmt::Display for MachineInfo<'_> {
for range in &r.regions {
writeln!(
f,
"MMIO DEVICE {:<4}: {}..{} {:<20} {:?}",
"MMIO DEVICE {:<11}: {}..{} {:<20} {:?}",
idx, range.start, range.end, r.name, r.compatible
)?;
}
Expand Down Expand Up @@ -330,8 +330,8 @@ impl<'dt> Visitor<'dt> for SocVisitorChildVisitor<'dt> {
Ok(())
}

fn visit_compatible(&mut self, mut strings: Strings<'dt>) -> Result<(), Self::Error> {
self.compatible = strings.collect::<Result<_,_>>()?;
fn visit_compatible(&mut self, strings: Strings<'dt>) -> Result<(), Self::Error> {
self.compatible = strings.collect::<Result<_, _>>()?;

Ok(())
}
Expand Down
4 changes: 2 additions & 2 deletions kernel/src/start.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ fn start(hartid: usize, boot_info: &'static loader_api::BootInfo) -> ! {
}

fn begin_hart_init(hartid: usize, fdt: *const u8) -> crate::Result<()> {
semihosting_logger::hartid::set(hartid);
logger::init_hart(hartid);
arch::trap_handler::init();

let minfo = unsafe { HartLocalMachineInfo::from_dtb(hartid, fdt)? };
Expand All @@ -70,7 +70,7 @@ fn begin_hart_init(hartid: usize, fdt: *const u8) -> crate::Result<()> {
}

fn init(boot_info: &'static loader_api::BootInfo, fdt: *const u8) -> crate::Result<()> {
semihosting_logger::init(LOG_LEVEL.to_level_filter());
logger::init(LOG_LEVEL.to_level_filter());

log::debug!("\n{boot_info}");

Expand Down
6 changes: 3 additions & 3 deletions kernel/src/time.rs
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ impl SystemTime {
(time_ns % NANOS_PER_SEC) as u32,
))
}

pub fn duration_since(&self, earlier: SystemTime) -> Result<Duration, SystemTimeError> {
if self >= &earlier {
Ok(self.0 - earlier.0)
Expand Down Expand Up @@ -306,7 +306,7 @@ mod tests {

#[ktest::test]
fn measure_and_timeout() {
let start_sys = SystemTime::now();
// let start_sys = SystemTime::now();
let start = Instant::now();

unsafe {
Expand All @@ -318,6 +318,6 @@ mod tests {
log::trace!("Time elapsed: {elapsed:?}");

assert_eq!(elapsed.as_secs(), 1);
assert_eq!(start_sys.elapsed().unwrap().as_secs(), 1)
// assert_eq!(start_sys.elapsed().unwrap().as_secs(), 1)
}
}
28 changes: 8 additions & 20 deletions kernel/src/vm/aspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ impl AddressSpace {
/// If the algorithm fails to find a suitable spot in the first attempt, it will have collected the
/// total number of candidate spots and retry with a new `target_index` in the range [0, candidate_spot_count)
/// which guarantees that a spot will be found as long as `candidate_spot_count > 0`.
fn find_spot(&mut self, layout: Layout, entropy: u8) -> VirtualAddress {
pub fn find_spot(&mut self, layout: Layout, entropy: u8) -> VirtualAddress {
// behaviour:
// - find the leftmost gap that satisfies the size and alignment requirements
// - starting at the root,
Expand Down Expand Up @@ -292,13 +292,8 @@ impl AddressSpace {

// see if there is a suitable gap between the start of the address space and the first mapping
if let Some(root) = self.tree.root().get() {
let gap_size = root.min_first_byte.sub_addr(self.address_range.start);
let aligned_gap = self.address_range.start.align_up(layout.align())
..self
.address_range
.start
.add(gap_size)
.align_down(layout.align());
let aligned_gap =
(self.address_range.start..root.min_first_byte).align_in(layout.align());
let spot_count = spots_in_range(layout, aligned_gap.clone());
candidate_spot_count += spot_count;
if target_index < spot_count {
Expand All @@ -322,10 +317,7 @@ impl AddressSpace {
continue;
}

let gap_base = left.max_last_byte;
let gap_size = node.range.end.sub_addr(left.max_last_byte);
let aligned_gap = gap_base.align_up(layout.align())
..gap_base.add(gap_size).align_down(layout.align());
let aligned_gap = (left.max_last_byte..node.range.end).align_in(layout.align());
let spot_count = spots_in_range(layout, aligned_gap.clone());

candidate_spot_count += spot_count;
Expand All @@ -340,10 +332,7 @@ impl AddressSpace {
if let Some(right) = node.links.right() {
let right = unsafe { right.as_ref() };

let gap_base = node.range.end;
let gap_size = right.min_first_byte.sub_addr(node.range.end);
let aligned_gap = gap_base.align_up(layout.align())
..gap_base.add(gap_size).align_down(layout.align());
let aligned_gap = (node.range.end..right.min_first_byte).align_in(layout.align());
let spot_count = spots_in_range(layout, aligned_gap.clone());

candidate_spot_count += spot_count;
Expand All @@ -366,9 +355,8 @@ impl AddressSpace {

// see if there is a suitable gap between the end of the last mapping and the end of the address space
if let Some(root) = self.tree.root().get() {
let gap_size = usize::MAX - root.max_last_byte.as_raw();
let aligned_gap = root.max_last_byte.align_up(layout.align())
..root.max_last_byte.add(gap_size).align_down(layout.align());
let aligned_gap =
(root.max_last_byte..self.address_range.end).align_in(layout.align());
let spot_count = spots_in_range(layout, aligned_gap.clone());
candidate_spot_count += spot_count;
if target_index < spot_count {
Expand Down Expand Up @@ -535,7 +523,7 @@ impl AddressSpace {
// // }
}

pub(crate) struct Batch {
pub struct Batch {
mmu: Arc<Mutex<mmu::AddressSpace>>,
range: Range<VirtualAddress>,
flags: mmu::Flags,
Expand Down
16 changes: 10 additions & 6 deletions kernel/src/vm/mapping.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use crate::vm::aspace::Batch;
use crate::vm::{PageFaultFlags, PagedVmo, Vmo, WiredVmo, FRAME_ALLOC};
use alloc::boxed::Box;
use alloc::string::String;
Expand All @@ -14,7 +15,6 @@ use mmu::{arch, AddressRangeExt, VirtualAddress};
use pin_project_lite::pin_project;
use sync::Mutex;
use wavltree::Side;
use crate::vm::aspace::Batch;

pin_project! {
pub struct Mapping {
Expand Down Expand Up @@ -47,7 +47,7 @@ pin_project! {
this.range.size()).unwrap(),
&mut flush
) {
panic_unwind::panic_in_drop!("failed to unmap {}: {err}", this.name);
log::error!("failed to unmap {}: {err}", this.name);
}
}
}
Expand Down Expand Up @@ -75,6 +75,8 @@ impl Mapping {
vmo_offset: usize,
name: String,
) -> Pin<Box<Self>> {
debug_assert!(range.start <= range.end);

Box::pin(Self {
links: wavltree::Links::default(),
min_first_byte: range.start,
Expand All @@ -92,11 +94,13 @@ impl Mapping {
pub fn map_range(&self, batch: &mut Batch, range: Range<VirtualAddress>) -> crate::Result<()> {
assert!(!range.is_empty());
assert!(self.range.start <= range.start && self.range.end >= range.end);

let phys = self.vmo.lookup_contiguous(self.vmo_offset..self.vmo_offset + range.size())?;


let phys = self
.vmo
.lookup_contiguous(self.vmo_offset..self.vmo_offset + range.size())?;

batch.append(range.start, (phys.start, phys.size()), self.flags)?;

Ok(())
}

Expand Down
Loading
Loading