Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(smartengine): add abstraction layer for WASM engine #3257

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
fb5b2fe
init work: make the public API clear
xxchan Feb 26, 2023
7a30006
implement filter transform
xxchan Feb 26, 2023
685ae36
wip
xxchan Apr 9, 2023
15686cc
make the public API clear
xxchan Feb 26, 2023
1937438
move wasmtime_engine to a different mod
xxchan Apr 15, 2023
6f1604e
fix wasi feature
xxchan Apr 15, 2023
6b88555
rename wasmtime-engine -> wasmtime
xxchan Apr 15, 2023
cd038a3
Merge branch 'master' into xxchan/clena
xxchan Apr 18, 2023
86326da
reorg
xxchan Apr 18, 2023
e959a9d
Merge branch 'xxchan/clena' into xxchan/wasmedge
xxchan Apr 18, 2023
f17b8ef
support init for wasmedge/common
xxchan Apr 18, 2023
8b11252
Merge branch 'master' into xxchan/wasmedge
xxchan Apr 18, 2023
11e1751
make SmartModuleInstance common
xxchan Apr 18, 2023
abd4fb6
make create_transform common & reorg trait imp to imp
xxchan Apr 18, 2023
66bcbdd
more movement
xxchan Apr 18, 2023
0132522
Merge branch 'master' into xxchan/wasmedge
xxchan Apr 29, 2023
da7a2e2
add other tests for wasmedge
xxchan Apr 29, 2023
26d9053
support agg for common/wasmedge
xxchan Apr 29, 2023
56de150
movement
xxchan Apr 29, 2023
14c4c3c
refactor wasmtime to use the common code
xxchan Apr 29, 2023
3d8f4bf
WasmTime -> Wasmtime
xxchan Apr 29, 2023
d0bb940
move transform unit tests to common
xxchan Apr 29, 2023
7924834
rename Wasmedge -> WasmEdge
xxchan Apr 29, 2023
f2f55e8
minor tweaks
xxchan Apr 29, 2023
e6f7e2e
change features
xxchan May 15, 2023
06d579f
rm wasmedge
xxchan May 31, 2023
6ff943b
Merge branch 'master' into xxchan/wasmedge
xxchan May 31, 2023
7e559c1
remove wasmedge
xxchan May 31, 2023
7d23e1a
fmt
xxchan May 31, 2023
f5650e3
fmt with group_imports = "StdExternalCrate"
xxchan May 31, 2023
fe98b8f
try to make diff smaller
xxchan May 31, 2023
136c572
make diff smaller
xxchan May 31, 2023
72f4ed7
clippy
xxchan May 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
try to make diff smaller
  • Loading branch information
xxchan committed May 31, 2023
commit fe98b8f391ecaab1aff1ce47b93aad947be66d89
45 changes: 41 additions & 4 deletions crates/fluvio-smartengine/src/engine/wasmtime/instance.rs
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
use std::sync::Arc;
use std::sync::{Arc, Mutex};

use anyhow::Result;
use fluvio_protocol::{Decoder, Encoder};
use tracing::debug;
use fluvio_smartmodule::dataplane::smartmodule::SmartModuleExtraParams;
use wasmtime::{AsContext, Caller, Extern, Instance, Module};
use wasmtime::{AsContext, Caller, Extern, Instance, Module, Memory};

use crate::engine::{
common::{WasmFn, WasmInstance},
error::EngineError,
wasmtime::memory::RecordsMemory,
};
use super::{memory::RecordsCallBack, state::WasmState};
use super::state::WasmState;

pub struct WasmtimeInstance {
instance: Instance,
Expand Down Expand Up @@ -123,3 +122,41 @@ impl WasmtimeInstance {
})
}
}

#[derive(Clone)]
pub struct RecordsMemory {
pub ptr: i32,
pub len: i32,
pub memory: Memory,
}

impl RecordsMemory {
pub fn copy_memory_from(&self, store: impl AsContext) -> Result<Vec<u8>> {
let mut bytes = vec![0u8; self.len as u32 as usize];
self.memory.read(store, self.ptr as usize, &mut bytes)?;
Ok(bytes)
}
}

pub struct RecordsCallBack(Mutex<Option<RecordsMemory>>);

impl RecordsCallBack {
pub(crate) fn new() -> Self {
Self(Mutex::new(None))
}

pub(crate) fn set(&self, records: RecordsMemory) {
let mut write_inner = self.0.lock().unwrap();
write_inner.replace(records);
}

pub(crate) fn clear(&self) {
let mut write_inner = self.0.lock().unwrap();
write_inner.take();
}

pub(crate) fn get(&self) -> Option<RecordsMemory> {
let reader = self.0.lock().unwrap();
reader.clone()
}
}
40 changes: 0 additions & 40 deletions crates/fluvio-smartengine/src/engine/wasmtime/memory.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
use std::sync::Mutex;

use wasmtime::*;
use anyhow::{anyhow, Error, Result};

Expand Down Expand Up @@ -56,41 +54,3 @@ pub(crate) fn copy_memory_to_instance(

Ok(guest_ptr_offset)
}

#[derive(Clone)]
pub struct RecordsMemory {
pub ptr: i32,
pub len: i32,
pub memory: Memory,
}

impl RecordsMemory {
pub fn copy_memory_from(&self, store: impl AsContext) -> Result<Vec<u8>> {
let mut bytes = vec![0u8; self.len as u32 as usize];
self.memory.read(store, self.ptr as usize, &mut bytes)?;
Ok(bytes)
}
}

pub struct RecordsCallBack(Mutex<Option<RecordsMemory>>);

impl RecordsCallBack {
pub(crate) fn new() -> Self {
Self(Mutex::new(None))
}

pub(crate) fn set(&self, records: RecordsMemory) {
let mut write_inner = self.0.lock().unwrap();
write_inner.replace(records);
}

pub(crate) fn clear(&self) {
let mut write_inner = self.0.lock().unwrap();
write_inner.take();
}

pub(crate) fn get(&self) -> Option<RecordsMemory> {
let reader = self.0.lock().unwrap();
reader.clone()
}
}
4 changes: 2 additions & 2 deletions crates/fluvio-smartengine/src/engine/wasmtime/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/// Implementations of the public API
mod imp;
pub use imp::{initialize_imp, SmartEngineImp, SmartModuleChainInstanceImp};
mod engine;
pub use engine::{initialize_imp, SmartEngineImp, SmartModuleChainInstanceImp};
/// Implementations of the traits in `common` for the Wasmtime engine
mod instance;
#[cfg(test)]
Expand Down