Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[OpenVINO-EP] Enable EP config options for VPU hardware #5119

Merged
merged 11 commits into from
Sep 14, 2020
Merged

[OpenVINO-EP] Enable EP config options for VPU hardware #5119

merged 11 commits into from
Sep 14, 2020

Conversation

smkarlap
Copy link
Contributor

Description: Adds two OpenVINO EP configuration options:-

  1. Enable Fast compilation of models for VPU hardware
  2. Select specific hardware device id as target for inference

Motivation and Context

  • Fast compilation is helpful for large models being initialized on slow CPU/low memory devices.
  • Enabling specific device id selection provides finer grain control over OpenVINO inference targets.

intel and others added 6 commits September 10, 2020 10:33
@smkarlap
Added config flags for VPU Fast Recompile
0dde6f2
@smkarlap
clean-up ifdefs
d487bab
@smkarlap
Add VPU Fast compile config option
7fb81c7 
Adds an option that enables Fast compilation of models to VPU
hardware specific format.
@smkarlap
Add config option to choose specific device id for inference
4a0bb65 
Inference of all subgraphs will be scheduled only on this device
even if other devices of the same type are available.
@smkarlap
Add Python API to list available device IDs
ce9d74c
@smkarlap
code cleanup
28717f3
@smkarlap smkarlap requested a review from a team as a code owner September 10, 2020 19:52
@smkarlap
Add second C/C++ API with settings string parameter
8ac5e98 
Adds an additional C/C++ API that allows passing multiple
key-value pairs for settings as a single string. Multiple
settings are delimited by '\n' while the key and value
within a setting are delimited by '|'.
@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@smkarlap
Append 'Ex' to the extended C/C++ API
2e09984
@smkarlap
Use set_providers Py API to set config options.
13bc9ab 
Uses Session.set_providers Python API to set EP runtime config
options as key/val pairs
Deprecated older module function definitions for config settings.
Updates documentation.
@smkarlap
Merge branch 'master' into manohar/vpu-config
a3e085b
@jywu-msft
Copy link
Member

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Linux OpenVINO CI Pipeline

@jywu-msft
Copy link
Member

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 8 pipeline(s).

@azure-pipelines
Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@jywu-msft
Copy link
Member

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Linux OpenVINO CI Pipeline

@jywu-msft
Copy link
Member

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 8 pipeline(s).

@azure-pipelines
Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@jywu-msft jywu-msft merged commit f7edf0a into microsoft:master Sep 14, 2020
@smkarlap smkarlap deleted the manohar/vpu-config branch September 16, 2020 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pranavsharma pranavsharma pranavsharma left review comments

@jywu-msft jywu-msft jywu-msft approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@smkarlap @jywu-msft @pranavsharma