Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support WasmEdge as an alternative engine #1

Draft
wants to merge 33 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
fb5b2fe
init work: make the public API clear
xxchan Feb 26, 2023
7a30006
implement filter transform
xxchan Feb 26, 2023
685ae36
wip
xxchan Apr 9, 2023
15686cc
make the public API clear
xxchan Feb 26, 2023
1937438
move wasmtime_engine to a different mod
xxchan Apr 15, 2023
6f1604e
fix wasi feature
xxchan Apr 15, 2023
6b88555
rename wasmtime-engine -> wasmtime
xxchan Apr 15, 2023
cd038a3
Merge branch 'master' into xxchan/clena
xxchan Apr 18, 2023
86326da
reorg
xxchan Apr 18, 2023
e959a9d
Merge branch 'xxchan/clena' into xxchan/wasmedge
xxchan Apr 18, 2023
f17b8ef
support init for wasmedge/common
xxchan Apr 18, 2023
8b11252
Merge branch 'master' into xxchan/wasmedge
xxchan Apr 18, 2023
11e1751
make SmartModuleInstance common
xxchan Apr 18, 2023
abd4fb6
make create_transform common & reorg trait imp to imp
xxchan Apr 18, 2023
66bcbdd
more movement
xxchan Apr 18, 2023
0132522
Merge branch 'master' into xxchan/wasmedge
xxchan Apr 29, 2023
da7a2e2
add other tests for wasmedge
xxchan Apr 29, 2023
26d9053
support agg for common/wasmedge
xxchan Apr 29, 2023
56de150
movement
xxchan Apr 29, 2023
14c4c3c
refactor wasmtime to use the common code
xxchan Apr 29, 2023
3d8f4bf
WasmTime -> Wasmtime
xxchan Apr 29, 2023
d0bb940
move transform unit tests to common
xxchan Apr 29, 2023
7924834
rename Wasmedge -> WasmEdge
xxchan Apr 29, 2023
f2f55e8
minor tweaks
xxchan Apr 29, 2023
e6f7e2e
change features
xxchan May 15, 2023
06d579f
rm wasmedge
xxchan May 31, 2023
6ff943b
Merge branch 'master' into xxchan/wasmedge
xxchan May 31, 2023
7e559c1
remove wasmedge
xxchan May 31, 2023
7d23e1a
fmt
xxchan May 31, 2023
f5650e3
fmt with group_imports = "StdExternalCrate"
xxchan May 31, 2023
fe98b8f
try to make diff smaller
xxchan May 31, 2023
136c572
make diff smaller
xxchan May 31, 2023
72f4ed7
clippy
xxchan May 31, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
init work: make the public API clear
  • Loading branch information
xxchan committed Apr 15, 2023
commit fb5b2fe2be03997469db74f8abf3ad5d0c53e374
151 changes: 151 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions crates/fluvio-smartengine/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@ description = "The official Fluvio SmartEngine"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[features]
engine = ["wasmtime"]
engine = ["wasmedge-engine", "wasmtime-engine"]
wasi = ["wasmtime-wasi", "engine"]
transformation = ["serde_json", "serde_yaml"]
default = ["engine"]

wasmtime-engine = ["wasmtime", "wasmtime-wasi"]
wasmedge-engine = ["wasmedge-sdk"]

[dependencies]
tracing ={ workspace = true }
Expand All @@ -27,6 +28,7 @@ wasmtime-wasi = { version = "7.0.0", optional = true }
serde = { version = "1", features = ['derive'] }
serde_json = { version = "1.0", optional = true }
serde_yaml = { version = "0.9", default-features = false, optional = true }
wasmedge-sdk = { version = "0.7", optional = true }

cfg-if = "1.0.0"
derive_builder = { workspace = true }
Expand Down
73 changes: 73 additions & 0 deletions crates/fluvio-smartengine/src/config.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
use derive_builder::Builder;
use fluvio_smartmodule::dataplane::smartmodule::{SmartModuleExtraParams};

const DEFAULT_SMARTENGINE_VERSION: i16 = 17;

/// Initial seed data to passed, this will be send back as part of the output
#[derive(Debug, Clone)]
#[non_exhaustive]
pub enum SmartModuleInitialData {
None,
Aggregate { accumulator: Vec<u8> },
}

impl SmartModuleInitialData {
pub fn with_aggregate(accumulator: Vec<u8>) -> Self {
Self::Aggregate { accumulator }
}
}

impl Default for SmartModuleInitialData {
fn default() -> Self {
Self::None
}
}

/// SmartModule configuration
#[derive(Builder)]
pub struct SmartModuleConfig {
#[builder(default, setter(strip_option))]
pub(crate) initial_data: SmartModuleInitialData,
#[builder(default)]
pub(crate) params: SmartModuleExtraParams,
// this will be deprecated in the future
#[builder(default, setter(into, strip_option))]
pub(crate) version: Option<i16>,
}

impl SmartModuleConfigBuilder {
/// add initial parameters
pub fn param(&mut self, key: impl Into<String>, value: impl Into<String>) -> &mut Self {
let mut new = self;
let mut params = new.params.take().unwrap_or_default();
params.insert(key.into(), value.into());
new.params = Some(params);
new
}
}

impl SmartModuleConfig {
pub fn builder() -> SmartModuleConfigBuilder {
SmartModuleConfigBuilder::default()
}

pub(crate) fn version(&self) -> i16 {
self.version.unwrap_or(DEFAULT_SMARTENGINE_VERSION)
}
}

#[cfg(feature = "transformation")]
impl From<crate::transformation::TransformationStep> for SmartModuleConfig {
fn from(step: crate::transformation::TransformationStep) -> Self {
Self {
initial_data: SmartModuleInitialData::None,
params: step
.with
.into_iter()
.map(|(k, v)| (k, v.into()))
.collect::<std::collections::BTreeMap<String, String>>()
.into(),
version: None,
}
}
}
Loading