Skip to content

DllMain is never run when given stdcall decorated export #135188

Open
@PARTYMANX

Description

I'm creating a Windows DLL in an admittedly goofy fashion: I have an executable that links a DLL and imports DllMain, using stdcall decorations (so in reality it's looking for _DllMain@12). This causes the DllMain function to not actually run when the DLL is linked. I made a dummy DllMain function that just returns false to prevent the linking program from running, making it obvious that my code is running, but the program is starting successfully. It's worth noting that I'm running into this when testing out replacing a C DLL with a new, Rust-based DLL. My C version natively exports _DllMain@12

Using the common wisdom of using #[no_mangle] on DllMain, it is failing to link with my executable because it does not export _DllMain@12. When modifying the executable to import DllMain, my own DllMain function runs, causing the program to fail to start (as intended, in this test case).

When testing with a program that instead uses LoadLibrary(), my code will run in both cases, indicating that there's different behavior for a DLL being linked and loaded programmatically. I'm not an expert on how DLLs are loaded, but it seems like there's an entrypoint function that is run when the DLL is linked, and the entrypoint will only run DllMain when its symbol is specifically DllMain instead of the also valid _DllMain@12.

I tried this code:

#[unsafe(export_name = "_DllMain@12")]
pub extern "stdcall" fn DllMain(_hinst_dll: usize, _fdw_reason: u32, _lp_reserved: usize) -> i32 {
    0
}

(crate-type is set to cdylib)

I expected to see this happen: When the DLL is linked, the program fails due to DllMain returning false

Instead, this happened: The DLL links successfully, with DllMain seemingly returning true

Meta

rustc --version --verbose:

$ rustc --version --verbose
rustc 1.83.0 (90b35a623 2024-11-26)
binary: rustc
commit-hash: 90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf
commit-date: 2024-11-26
host: x86_64-pc-windows-msvc
release: 1.83.0
LLVM version: 19.1.1

Metadata

Assignees

No one assigned

    Labels

    A-dynamic-libraryArea: Dynamic/Shared LibrariesA-linkageArea: linking into static, shared libraries and binariesC-bugCategory: This is a bug.O-windowsOperating system: Windowsneeds-triageThis issue may need triage. Remove it if it has been sufficiently triaged.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions