wasi-nn: update openvino backend to v0.7.2, use API 2.0 #8790
Conversation
|
Converting this to draft temporarily, plan to bump openvino crate to a newer version than 0.7.1 |
alexcrichton
requested review from
abrown
and removed request for
a team and
fitzgen
rahulchaphalkar
force-pushed
the
update-ov-version
branch
from
2b82183 to
c098e32
Compare
abrown
changed the title
rahulchaphalkar
force-pushed
the
update-ov-version
branch
from
0fa3445 to
f515afe
Compare
|
@abrown I pushed some changes to fix failures due to some incompatibilities with |
|
I already did... where did that commit go? |
| @@ -157,24 +148,51 @@ impl From<SetupError> for BackendError { | |||
|
|
|||
| /// Return the execution target string expected by OpenVINO from the | |||
| /// `ExecutionTarget` enum provided by wasi-nn. | |||
| fn map_execution_target_to_string(target: ExecutionTarget) -> &'static str { | |||
| fn map_execution_target_to_string(target: ExecutionTarget) -> DeviceType<'static> { | |||
There was a problem hiding this comment.
If possible, let's rewrite this as impl From<ExecutionTarget> for DeviceType<'static>. If not possible then at least change the function name and documentation.
| ExecutionTarget::Tpu => unimplemented!("OpenVINO does not support TPU execution targets"), | ||
| } | ||
| } | ||
|
|
||
| /// Return OpenVINO's precision type for the `TensorType` enum provided by | ||
| /// wasi-nn. | ||
| fn map_tensor_type_to_precision(tensor_type: TensorType) -> openvino::Precision { | ||
| fn map_tensor_type_to_precision(tensor_type: TensorType) -> openvino::ElementType { |
There was a problem hiding this comment.
Similarly, this could be impl From<TensorType> for ElementType.
| } | ||
|
|
||
| /// Return the `TensorType` enum provided by wasi-nn for OpenVINO's precision type | ||
| fn map_precision_to_tensor_type(element_type: openvino::ElementType) -> TensorType { |
There was a problem hiding this comment.
This can be a TryFrom since it is more useful to return an error to the user than crash Wasmtime with todo!.
| ElementType::I32 => TensorType::I32, | ||
| ElementType::I64 => TensorType::I64, | ||
| ElementType::Bf16 => TensorType::Bf16, | ||
| ElementType::Undefined => todo!("not yet supported in wasi-nn `TensorType`"), |
There was a problem hiding this comment.
I think this could get cleaned up to something like: ElementType::Undefined | ElementType::Dynamic | ... => Err(BackendError::SomeNewVariant("{:?} is not yet supported in the openvino backend", element_type))
| self.1.set_blob(&input_name, &blob)?; | ||
| let shape = Shape::new(&dimensions)?; | ||
| let mut new_tensor = OvTensor::new(precision, &shape)?; | ||
| let buffer = new_tensor.get_raw_data_mut()?; |
There was a problem hiding this comment.
If no need to specify layout (of input tensor), and there is no PrePostProcessing in the whole file, Does it mean wasm code has to make sure the input tensor matches the requirement of model input? (I assume openvino models requires "NHWC" input)
There was a problem hiding this comment.
Hi Liang, yes this requires input tensor layout match the expected layout for model input.
openvino.rs.