Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OVEP: Tensor caching fix #22218

Merged
merged 4 commits into from
Sep 25, 2024
Merged

OVEP: Tensor caching fix #22218

merged 4 commits into from
Sep 25, 2024

Conversation

saurabhkale17
Copy link
Contributor

Description

  1. changing the emplace to [] that does have a difference, emplace will only create a new entry if it doesn't already exist in the map
  2. change the logic of the caching lookup to key off of input/output names instead of ort raw ptrs.
  3. changes OV tensor creation for CPU allocated input/output ORT tensors. The CPU allocated input/output tensor path was re-allocating OV tensors based on the ORT input/output tensors. So we'd get 2 copies: ORT input/output tensor -> OV tensor (OVEP) -> NPU Tensor (NPU plugin).

@saurabhkale17 saurabhkale17 changed the title Saurabh/fix regression OVEP: Tensor caching fix Sep 25, 2024
@sfatimar
Copy link
Contributor

@jywu-msft @adrianlizarraga

@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@adrianlizarraga
Copy link
Contributor

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, Linux QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 9 pipeline(s).

@adrianlizarraga
Copy link
Contributor

/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows ARM64 QNN CI Pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, Windows x64 QNN CI Pipeline, Linux MIGraphX CI Pipeline, Big Models

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 8 pipeline(s).

@adrianlizarraga
Copy link
Contributor

/azp run Linux Android Emulator QNN CI Pipeline, Windows GPU CUDA CI Pipeline, Windows GPU DML CI Pipeline, Windows GPU Doc Gen CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 4 pipeline(s).

@jywu-msft jywu-msft merged commit 4d6019f into microsoft:main Sep 25, 2024
72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jywu-msft jywu-msft jywu-msft left review comments

@javier-intel javier-intel javier-intel left review comments

@adrianlizarraga adrianlizarraga adrianlizarraga approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@saurabhkale17 @sfatimar @jywu-msft @adrianlizarraga @javier-intel @ericcraw