StaticLLMPipeline: Decide when to enable NPUW_DQ_FULL property #1258
Conversation
smirnov-alexey
changed the title
src/cpp/CMakeLists.txt
Outdated
| @@ -71,7 +71,7 @@ target_include_directories(${TARGET_NAME} | |||
|
|
|||
| target_include_directories(${TARGET_NAME} SYSTEM PRIVATE "${safetensors.h_SOURCE_DIR}") | |||
|
|
|||
| target_link_libraries(${TARGET_NAME} PUBLIC openvino::runtime PRIVATE openvino::threading nlohmann_json::nlohmann_json jinja2cpp) | |||
| target_link_libraries(${TARGET_NAME} PUBLIC openvino::runtime openvino::runtime::dev PRIVATE openvino::threading nlohmann_json::nlohmann_json jinja2cpp) | |||
There was a problem hiding this comment.
Required for ov::internal::supported_properties. Is there any better way?
There was a problem hiding this comment.
it breaks OpenVINO GenAI compilation against public OpenVINO package where Dev part is not available.
src/cpp/CMakeLists.txt
Outdated
| @@ -71,7 +71,7 @@ target_include_directories(${TARGET_NAME} | |||
|
|
|||
| target_include_directories(${TARGET_NAME} SYSTEM PRIVATE "${safetensors.h_SOURCE_DIR}") | |||
|
|
|||
| target_link_libraries(${TARGET_NAME} PUBLIC openvino::runtime PRIVATE openvino::threading nlohmann_json::nlohmann_json jinja2cpp) | |||
| target_link_libraries(${TARGET_NAME} PUBLIC openvino::runtime openvino::runtime::dev PRIVATE openvino::threading nlohmann_json::nlohmann_json jinja2cpp) | |||
There was a problem hiding this comment.
it breaks OpenVINO GenAI compilation against public OpenVINO package where Dev part is not available.
| bool compiler_dq = false; | ||
| const auto device_caps = core.get_property("NPU", ov::device::capabilities); | ||
| if (std::find(device_caps.begin(), device_caps.end(), | ||
| "COMPILER_DYNAMIC_QUANTIZATION") != device_caps.end()) { | ||
| compiler_dq = true; | ||
| } |
There was a problem hiding this comment.
| bool compiler_dq = false; | |
| const auto device_caps = core.get_property("NPU", ov::device::capabilities); | |
| if (std::find(device_caps.begin(), device_caps.end(), | |
| "COMPILER_DYNAMIC_QUANTIZATION") != device_caps.end()) { | |
| compiler_dq = true; | |
| } | |
| const bool compiler_dq = caps.end() != std::find(caps.begin(), caps.end(), "COMPILER_DYNAMIC_QUANTIZATION") |
| std::string arch; | ||
| int64_t max_tiles; | ||
| bool compiler_dq; |
There was a problem hiding this comment.
I think it is worth to have default values here, probably even incorrect (e.g., 0 as max_tiles)
|
Please, add |
|
Property to be added in openvinotoolkit/openvino#27789 |
|
@ilya-lavrenov what branch should I target for OV24.6? Or adding |
|
…inotoolkit#1258) Based on (yet to be) supported OV properties from the NPU Plugin enable NPUW_DQ_FULL. releases/2024/5 mirror: openvinotoolkit#1272 Dependencies * openvinotoolkit/openvino#27678 needs to be merged first * openvinotoolkit/openvino#27789
Based on (yet to be) supported OV properties from the NPU Plugin enable NPUW_DQ_FULL.
releases/2024/5 mirror: #1272
Dependencies
E-148336