[QDQ] Add shared NodeUnit class #10052
Conversation
onnxruntime/core/providers/nnapi/nnapi_builtin/builders/op_builder.cc
Outdated
Show resolved
Hide resolved
…lder.cc Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
| namespace onnxruntime { | ||
| namespace nnapi { | ||
|
|
||
| using onnxruntime::NodeUnit; |
There was a problem hiding this comment.
why are these using declarations needed? I didn't see any unqualified usages of vector or string in this file, and the NodeUnit usage is in the onnxruntime namespace
There was a problem hiding this comment.
similar comment for other source files
There was a problem hiding this comment.
I got compile error if I don't do this, unless I add onnxruntime:: namespace to the NodeUnit
There was a problem hiding this comment.
I see, is there another NodeUnit somewhere? just curious
There was a problem hiding this comment.
There is no other NodeUnit defined, the complier is trying to find onnxruntime::nnapi::NodeUnit, added using for simplicity, otherwise it will be onnxruntime::NodeUnit every where
There was a problem hiding this comment.
that's odd - how is something like onnxruntime::GraphViewer working?
onnxruntime/core/providers/nnapi/nnapi_builtin/builders/model_builder.cc
Outdated
Show resolved
Hide resolved
…runtime into gwang-msft/add_nodee_unit
|
/azp run orttraining-amd-gpu-ci-pipeline |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Description: Add NodeUnit class
Motivation and Context
To avoid excessively large PR, the NNAPI QDQ integration is splitted into the following small tasks
1. Add shared NodeUnit class (this PR)
2. Move NNAPI EP to use NodeUnit IODef for non-QDQ ops
3. Add shared QDQ selectors
4. Hookup NNAPI GetCapability/Compile with shared QDQ selectors
5. Enable QDQ for ops with QLinear version (QLinear[Conv/Matmul/Pool/...])
6. Enable QDQ for ops without QLinear Version
This is step 1 of the tasks, we add the shared NodeUnit and move NNAPI builder interface to take NodeUnit, the NNAPI builders still work as before, we have not yet converted the quantized input of some QLinear ops to quant_params, which will be covered in step 2 (there are some code (later removed from
onnxruntime/core/providers/shared/node_unit/node_unit.cc) in this commit of the PR)The shared NodeUnit class will be shared between a single Node or a QDQ group, which will both be treated as a unit with necessary functions, the main difference between a Node interface and a NodeUnit interface is the IODef which embed the input with its quantization params (scale and ZP), which defines a consistent way to retrieve quantization information for input and output