Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix delay load for WebGPU EP and DML EP #23111

Merged
merged 11 commits into from
Dec 19, 2024
Merged

Fix delay load for WebGPU EP and DML EP #23111

merged 11 commits into from
Dec 19, 2024

Conversation

fs-eire
Copy link
Contributor

@fs-eire fs-eire commented Dec 15, 2024

Description

This change fixes the DLL delay load problem for the WebGPU EP and DirectML EP. See detailed explanation below.

Problem

When onnxruntime.dll uses delay loading for its dependencies, the dependencies are loaded using LoadLibraryEx(), which search the directory of process (.exe) instead of this library (onnxruntime.dll). This is a problem for usages of Node.js binding and python binding, because Windows will try to find the dependencies in the directory of node.exe or python.exe, which is not the directory of onnxruntime.dll.

There was previous attempt to fix this by loading DirectML.dll in the initialization of onnxruntime nodejs binding, which works for DML EP but is not a good solution because it does not really "delay" the load.

For WebGPU, the situation became worse because webgpu_dawn.dll depends on dxil.dll and dxcompiler.dll, which are explicitly dynamically loaded in the code using LoadLibraryA(). This has the same problem of the DLL search.

Solutions

For onnxruntime.dll loading its direct dependencies, it can be resolved by set the __pfnDliNotifyHook2 hook to load from an absolute path that constructed from the onnxruntime.dll folder and the DLL name.

For webgpu_dawn.dll loading dxil.dll and dxcompiler.dll, since they are explicitly loaded in the code, the hook does not work. Instead, it can be resolved by using WIN32 API SetDllDirectory() to add the onnxruntime.dll folder to the search path. preloading the 2 DLLs from the onnxruntime.dll folder .

@fs-eire fs-eire marked this pull request as ready for review December 16, 2024 23:16
@fs-eire fs-eire requested a review from a team as a code owner December 16, 2024 23:16
@fs-eire fs-eire requested review from snnn and guschmue December 16, 2024 23:17
Copy link
Member

@snnn snnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@fs-eire fs-eire merged commit 8680244 into main Dec 19, 2024
93 checks passed
@fs-eire fs-eire deleted the fs-eire/dl2 branch December 19, 2024 18:23
guschmue pushed a commit that referenced this pull request Dec 20, 2024
### Description

This change fixes the DLL delay load problem for the WebGPU EP and
DirectML EP. See detailed explanation below.

### Problem

When onnxruntime.dll uses delay loading for its dependencies, the
dependencies are loaded using `LoadLibraryEx()`, which search the
directory of process (.exe) instead of this library (onnxruntime.dll).
This is a problem for usages of Node.js binding and python binding,
because Windows will try to find the dependencies in the directory of
node.exe or python.exe, which is not the directory of onnxruntime.dll.

There was previous attempt to fix this by loading DirectML.dll in the
initialization of onnxruntime nodejs binding, which works for DML EP but
is not a good solution because it does not really "delay" the load.

For WebGPU, the situation became worse because webgpu_dawn.dll depends
on dxil.dll and dxcompiler.dll, which are explicitly dynamically loaded
in the code using `LoadLibraryA()`. This has the same problem of the DLL
search.

### Solutions

For onnxruntime.dll loading its direct dependencies, it can be resolved
by set the [`__pfnDliNotifyHook2`
hook](https://learn.microsoft.com/en-us/cpp/build/reference/understanding-the-helper-function?view=msvc-170#structure-and-constant-definitions)
to load from an absolute path that constructed from the onnxruntime.dll
folder and the DLL name.

For webgpu_dawn.dll loading dxil.dll and dxcompiler.dll, since they are
explicitly loaded in the code, the hook does not work. Instead, it can
be resolved by ~~using WIN32 API `SetDllDirectory()` to add the
onnxruntime.dll folder to the search path.~~ preloading the 2 DLLs from
the onnxruntime.dll folder .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants