Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: migrate disk collection code off of heim, remove heim #1064

Merged
merged 39 commits into from
Apr 10, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
acd79e5
refactor: fix some refresh code
ClementTsang Mar 9, 2023
e58746d
remove async from the freebsd code
ClementTsang Mar 11, 2023
7754d02
some file/implementation organization
ClementTsang Mar 14, 2023
577bf72
more restructuring
ClementTsang Mar 16, 2023
5e1592c
Some other fixes
ClementTsang Mar 19, 2023
b2594ce
remove futures
ClementTsang Mar 19, 2023
48158e7
ready for some big changes?
ClementTsang Mar 23, 2023
4d5f250
big changes
ClementTsang Mar 26, 2023
69675dc
linux io + reads
ClementTsang Apr 1, 2023
ecf0ac2
use lossy conversion for mount point
ClementTsang Apr 1, 2023
8724780
add windows refresh
ClementTsang Apr 1, 2023
794eda3
so long heim, and thanks for all the fish
ClementTsang Apr 1, 2023
c53a32e
fix filter behaviour, remove string allocation when reading lines
ClementTsang Apr 3, 2023
042c530
rename unix -> system for more accurate file struct representation
ClementTsang Apr 4, 2023
7e0a2be
fix freebsd
ClementTsang Apr 4, 2023
37f4fd7
port generic unix partition code
ClementTsang Apr 5, 2023
b10d124
add bindings and fix errors
ClementTsang Apr 5, 2023
ad1d8f9
finish macOS bindings for I/O
ClementTsang Apr 5, 2023
9c8df52
disable conform check, this seems to... make disk I/O work on macOS?????
ClementTsang Apr 5, 2023
c027da9
fix linux
ClementTsang Apr 5, 2023
32e38ae
add safety comments
ClementTsang Apr 5, 2023
2628254
more comments
ClementTsang Apr 5, 2023
72485bc
update changelog
ClementTsang Apr 5, 2023
6d6bec0
changelog
ClementTsang Apr 5, 2023
096190d
We're going full 0.9.0 for this
ClementTsang Apr 5, 2023
e92cba7
update lock
ClementTsang Apr 5, 2023
c396659
fix some typing
ClementTsang Apr 6, 2023
eff2f16
bleh
ClementTsang Apr 6, 2023
461cde4
some file management
ClementTsang Apr 6, 2023
7f0c669
hoist out get_disk_usage
ClementTsang Apr 6, 2023
12559df
fix some stuff for Windows
ClementTsang Apr 6, 2023
30ea987
typing and remove dead code allow lint
ClementTsang Apr 6, 2023
0f4287b
unify typing
ClementTsang Apr 6, 2023
661c578
fix
ClementTsang Apr 6, 2023
111133c
fix 2
ClementTsang Apr 6, 2023
a30aac1
macOS fix
ClementTsang Apr 7, 2023
4d22589
Add bindings file for windows
ClementTsang Apr 9, 2023
0765343
add windows implementation
ClementTsang Apr 10, 2023
e5d241a
fix macos
ClementTsang Apr 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
finish macOS bindings for I/O
  • Loading branch information
ClementTsang committed Apr 9, 2023
commit ad1d8f9953d5e51ddea061cc3b7a2b9ea42aeb42
19 changes: 15 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ libc = "0.2.141"
procfs = { version = "0.15.1", default-features = false }

[target.'cfg(target_os = "macos")'.dependencies]
core-foundation = "0.9.3"
mach2 = "0.4.1"

[target.'cfg(target_os = "windows")'.dependencies]
Expand Down
24 changes: 6 additions & 18 deletions src/app/data_harvester/disks/system/linux/io_counters.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,8 @@ const DISK_SECTOR_SIZE: u64 = 512;
#[derive(Debug, Default)]
pub struct IoCounters {
name: String,
read_count: u64,
read_merged_count: u64,
read_bytes: u64,
read_time_secs: u64,
write_count: u64,
write_merged_count: u64,
write_bytes: u64,
write_time_secs: u64,
}

impl IoCounters {
Expand Down Expand Up @@ -71,26 +65,20 @@ impl FromStr for IoCounters {

let name = next_part(&mut parts)?.to_string();

let read_count = next_part(&mut parts)?.parse()?;
let read_merged_count = next_part(&mut parts)?.parse()?;
let _read_count = next_part(&mut parts)?.parse()?;
let _read_merged_count = next_part(&mut parts)?.parse()?;
let read_bytes = next_part(&mut parts)?.parse::<u64>()? * DISK_SECTOR_SIZE;
let read_time_secs = next_part(&mut parts)?.parse()?;
let _read_time_secs = next_part(&mut parts)?.parse()?;

let write_count = next_part(&mut parts)?.parse()?;
let write_merged_count = next_part(&mut parts)?.parse()?;
let _write_count = next_part(&mut parts)?.parse()?;
let _write_merged_count = next_part(&mut parts)?.parse()?;
let write_bytes = next_part(&mut parts)?.parse::<u64>()? * DISK_SECTOR_SIZE;
let write_time_secs = next_part(&mut parts)?.parse()?;
let _write_time_secs = next_part(&mut parts)?.parse()?;

Ok(IoCounters {
name,
read_count,
read_merged_count,
read_bytes,
read_time_secs,
write_count,
write_merged_count,
write_bytes,
write_time_secs,
})
}
}
Expand Down
37 changes: 31 additions & 6 deletions src/app/data_harvester/disks/system/macos/io_counters.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,15 @@

use std::ffi::OsStr;

use anyhow::bail;

use super::io_kit::{self, get_dict, get_disks, get_i64, get_string};

#[derive(Debug, Default)]
pub struct IoCounters {
name: String,

read_count: u64,
read_bytes: u64,
read_time_secs: u64,
write_count: u64,
write_bytes: u64,
write_time_secs: u64,
}

impl IoCounters {
Expand All @@ -28,7 +27,33 @@ impl IoCounters {
}
}

fn get_device_io(device: io_kit::IoObject) -> anyhow::Result<IoCounters> {
let parent = device.service_parent()?;

if parent.conforms_to_block_storage_driver() {
let disk_props = device.properties()?;
let parent_props = parent.properties()?;

let name = get_string(&disk_props, "BSD Name")?;
let stats = get_dict(&parent_props, "Statistics")?;

let read_bytes = get_i64(&stats, "Bytes (Read)")? as u64;
let write_bytes = get_i64(&stats, "Bytes (Write)")? as u64;

// let read_count = stats.get_i64("Operations (Read)")? as u64;
// let write_count = stats.get_i64("Operations (Write)")? as u64;

Ok(IoCounters {
name,
read_bytes,
write_bytes,
})
} else {
bail!("{parent:?}, the parent of {device:?} does not conform to IOBlockStorageDriver")
}
}

/// Returns an iterator of disk I/O stats. Pulls data through IOKit.
pub fn io_stats() -> anyhow::Result<Vec<anyhow::Result<IoCounters>>> {
Ok(vec![])
Ok(get_disks()?.map(get_device_io).collect())
}
10 changes: 10 additions & 0 deletions src/app/data_harvester/disks/system/macos/io_kit.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
mod bindings;

mod io_iterator;
pub use io_iterator::*;

mod io_object;
pub use io_object::*;

mod io_disks;
pub use io_disks::get_disks;
59 changes: 59 additions & 0 deletions src/app/data_harvester/disks/system/macos/io_kit/bindings.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
//! C FFI bindings for [IOKit](https://developer.apple.com/documentation/iokit/).
//!
//! Based on [heim](https://github.com/heim-rs/heim/blob/master/heim-common/src/sys/macos/iokit/io_master_port.rs)
//! and [sysinfo's implementation](https://github.com/GuillaumeGomez/sysinfo/blob/master/src/apple/macos/ffi.rs).
//!
//! Ideally, we can remove this if sysinfo ever gains disk I/O capabilities.

use core_foundation::base::{mach_port_t, CFAllocatorRef};
use core_foundation::dictionary::CFMutableDictionaryRef;

use libc::c_char;
use mach2::port::MACH_PORT_NULL;
use mach2::{boolean::boolean_t, kern_return::kern_return_t};

#[allow(non_camel_case_types)]
pub type io_object_t = mach_port_t;

#[allow(non_camel_case_types)]
pub type io_iterator_t = io_object_t;
#[allow(non_camel_case_types)]
pub type io_registry_entry_t = io_object_t;

pub type IOOptionBits = u32;

/// See https://github.com/1kc/librazermacos/pull/27#issuecomment-1042368531.
#[allow(non_upper_case_globals)]
pub const kIOMasterPortDefault: mach_port_t = MACH_PORT_NULL;

#[allow(non_upper_case_globals)]
pub const kIOServicePlane: &str = "IOService\0";

#[allow(non_upper_case_globals)]
pub const kIOMediaClass: &str = "IOMedia\0";

// See [here](https://developer.apple.com/documentation/iokit) for more details.
extern "C" {

pub fn IOServiceGetMatchingServices(
mainPort: mach_port_t, matching: CFMutableDictionaryRef, existing: *mut io_iterator_t,
) -> kern_return_t;

pub fn IOServiceMatching(name: *const c_char) -> CFMutableDictionaryRef;

pub fn IOIteratorNext(iterator: io_iterator_t) -> io_object_t;

pub fn IOObjectRelease(obj: io_object_t) -> kern_return_t;

pub fn IORegistryEntryGetParentEntry(
entry: io_registry_entry_t, plane: *const libc::c_char, parent: *mut io_registry_entry_t,
) -> kern_return_t;

pub fn IOObjectConformsTo(object: io_object_t, className: *const libc::c_char) -> boolean_t;

pub fn IORegistryEntryCreateCFProperties(
entry: io_registry_entry_t, properties: *mut CFMutableDictionaryRef,
allocator: CFAllocatorRef, options: IOOptionBits,
) -> kern_return_t;

}
24 changes: 24 additions & 0 deletions src/app/data_harvester/disks/system/macos/io_kit/io_disks.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
use anyhow::bail;

use mach2::kern_return;

use super::{bindings::*, IoIterator};

pub fn get_disks() -> anyhow::Result<IoIterator> {
let mut media_iter: io_iterator_t = 0;

// SAFETY: This is a safe syscall,
let result = unsafe {
IOServiceGetMatchingServices(
kIOMasterPortDefault,
IOServiceMatching(kIOMediaClass.as_ptr().cast()),
&mut media_iter,
)
};

if result == kern_return::KERN_SUCCESS {
Ok(media_iter.into())
} else {
bail!("IOServiceGetMatchingServices failed, error code {result}");
}
}
51 changes: 51 additions & 0 deletions src/app/data_harvester/disks/system/macos/io_kit/io_iterator.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
//! Based on [heim's](https://github.com/heim-rs/heim/blob/master/heim-common/src/sys/macos/iokit/io_iterator.rs).
//! implementation.

use std::ops::{Deref, DerefMut};

use mach2::kern_return;

use super::{bindings::*, io_object::IoObject};

/// Safe wrapper around the IOKit `io_iterator_t` type.
#[derive(Debug)]
pub struct IoIterator(io_iterator_t);

impl From<io_iterator_t> for IoIterator {
fn from(iter: io_iterator_t) -> IoIterator {
IoIterator(iter)
}
}

impl Deref for IoIterator {
type Target = io_iterator_t;

fn deref(&self) -> &Self::Target {
&self.0
}
}

impl DerefMut for IoIterator {
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.0
}
}

impl Iterator for IoIterator {
type Item = IoObject;

fn next(&mut self) -> Option<Self::Item> {
// Basically, we just stop when we hit 0.
match unsafe { IOIteratorNext(self.0) } {
0 => None,
io_object => Some(IoObject::from(io_object)),
}
}
}

impl Drop for IoIterator {
fn drop(&mut self) {
let result = unsafe { IOObjectRelease(self.0) };
assert_eq!(result, kern_return::KERN_SUCCESS);
}
}
Loading