Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling Hardware Acceleration on RDP #7684

Merged
merged 2 commits into from
Aug 11, 2023
Merged

Conversation

Kuldeep-MS
Copy link
Member

@Kuldeep-MS Kuldeep-MS commented Mar 29, 2023

Fixes #3215

Description

Enabling Hardware Acceleration on RDP for exes which can opt-in.

Customer Impact

Without this fix, customers won't be able to get the benefits of Hardware Acceleration on RDP.

Regression

No

Testing

In progress

Risk

Risk is Low as the applications which want to have hardware acceleration on RDP need to Opt-in by setting AppContext Switch (Switch.System.Windows.Media.EnableHardwareAccelerationForRdp) to true.

Microsoft Reviewers: Open in CodeFlow

@Kuldeep-MS Kuldeep-MS requested a review from a team as a code owner March 29, 2023 17:34
@ghost ghost assigned Kuldeep-MS Mar 29, 2023
@ghost ghost added the PR metadata: Label to tag PRs, to facilitate with triage label Mar 29, 2023
@ghost ghost requested review from dipeshmsft and singhashish-wpf March 29, 2023 17:34
@batzen
Copy link
Contributor

batzen commented Apr 4, 2023

I don't think using the registry and requiring the full exe path is a good solution.
How am i supposed to use this properly in apps that don't have an installer or during development?

@Kuldeep-MS
Copy link
Member Author

@batzen - Yes, you're correct. While the registry solution may not be suitable for all users, we have discovered that several switches in the unmanaged code rely on this approach. Link

 

Although using AppContextSwitch might have been a better option, we need to investigate its viability in the unmanaged part of WPF.

@Kuldeep-MS Kuldeep-MS force-pushed the enableHwAcc branch 3 times, most recently from 509c977 to 7d5766f Compare May 3, 2023 07:29
@JLucGarnier
Copy link

Hello,
what WPF version will embed the fix to this issue?
do you have any rough estimated time to get a public access to this version?
Thank you.

JLuc

@pchaurasia14
Copy link
Member

@JLucGarnier - We are targeting this for .NET 8. We're only left with the last steps here - To integrate an opt-in switch mechanism.
The current challenge is the integration of AppContextSwitches which flows back to unmanaged part of the codebase.
We are working to see how can we get this implemented asap.

@Kuldeep-MS Kuldeep-MS force-pushed the enableHwAcc branch 2 times, most recently from 92bfe6c to d251a99 Compare June 29, 2023 11:38
@Kuldeep-MS Kuldeep-MS changed the title [WIP] Enabling Hardware Acceleration on RDP Enabling Hardware Acceleration on RDP Jun 30, 2023
@Kuldeep-MS Kuldeep-MS force-pushed the enableHwAcc branch 2 times, most recently from c68c959 to 5a9517a Compare June 30, 2023 02:22
@Kuldeep-MS Kuldeep-MS force-pushed the enableHwAcc branch 2 times, most recently from 058d38a to 75d8c7c Compare July 4, 2023 14:49
Copy link
Contributor

@miloush miloush left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearly this needs to go through DRT in RDP.

I guess the biggest question is why is this opt-in rather than opt-out?

@miloush
Copy link
Contributor

miloush commented Jul 4, 2023

The idea that no display = no hw acceleration is also questionable to me.

@miloush
Copy link
Contributor

miloush commented Jul 4, 2023

The other flag that triggers m_fNonLocalDevicePresent is DISPLAY_DEVICE_MIRRORING_DRIVER. If Windows 7 is still supported OS for .NET 8, it might be worth checking things work.

A relevant documentation (reason for m_fNonLocalDevicePresent - XPDM) is https://learn.microsoft.com/en-us/windows/win32/direct3d9/xpdm-vs-wddm. It states that the creation of a Direct3D device will fail. Could we not just optimistically try to create a Direct3D device and if that fails, fallback to software rendering, rather than guessing what might work? In fact since XPDM is no longer allowed, this check seems to be completely unnecessary. Why are we introducing a switch for it?

@pchaurasia14
Copy link
Member

@miloush - Our rationale for making this change opt-in vs opt-out is the lack of appropriate test coverage in this area. There could be use-cases where such feature may start breaking applications post upgrade. Hence we are inclined to keep this opt-in for now and work on improving the tests around this area first.

Once we have that we can decide to turn it on by default for all users (in future .NET releases).

@miloush
Copy link
Contributor

miloush commented Jul 11, 2023

@pchaurasia14 While I understand the concern, I do not find an opt-in justified or beneficial enough in this case. You also cannot be adding compatibility flags for every change, it adds maintainability costs to the future.

The DirectX documentation states that creating the surface under RDP is now always supported. You have as much testing as you have for any other feature by running the DTS in a remote desktop session.

There already exist several opt out of hardware acceleration options for users:

  1. Global, using registry setting
  2. Process-wide using ProcessRenderMode which also documents these 3 options
  3. Per window using RenderMode

Hardware acceleration should be the normal state of things, and what we hope to have going forward. You are looking for more testing, but I am worried you will not get enough coverage by real-world applications if this is an opt in.

As indicated in the original issue, this change has been analysed several times in the past by @vatsan-madhavan and you are not going to get anything better than that. He also suggested running regression tests in RDP for verification and no reason for a flag in .NET (though he supported one for .NET Framework).

If you are not comfortable making this change without a flag, it sounds to me like an opt out rather than opt in would be a good compromise and with alignment of the other options to opt out of hw rendering.

In general, how do you decide whether to go for opt in or opt out, are there any criteria?

@pchaurasia14
Copy link
Member

@miloush - Fair points. Currently, our criteria is the higher number of unknowns in the code paths and the approaching timeline for .NET8 release. The change in the current PR is at a fairly low level which creates higher risk of applications failing to start if we have missed something. All the changes in the current PR were made on the issues we encountered in sample application (that uses multiple animations). With the limited coverage, we don't know if the code paths in question are being executed for any other use cases.

Also, as we have more (and better) tests around this, we will revisit this and turn on the flag for next releases (.NET9). We'll create a tracker issue for this. Having an opt-in gives us the room for experiment without causing disruption to people who would be migrating to newer .NET version.

@vatsan-madhavan
Copy link
Member

vatsan-madhavan commented Jul 11, 2023

Given that .NET is a side by side update (and not in-place) it makes sense that quirks like these, if they must exist, are either opt-outs or have a well understood compat-breaking scenario that's being guarded against.

Opt-ins are more naturally suited for .NET Framework, where fixes are in-place updates.

That said, if .NET 8 must get this change with an opt-in, I'd recommend making it the default in .NET 9 code base immediately (with optionally leaving the quirk in as an opt-out). Improvements like these should become default for wider adoption, and the earlier they're made default in .NET 9 the better it is for gaining confidence through community feedback etc.

@JeremyKuhne
Copy link
Member

On the topic of opt-in vs. opt-out, in WinForms we lean towards opt-out much as described by @vatsan-madhavan
#7684 (comment). Everything is case-by-case, of course, but we want the better experience to be the default for apps on .NET (core). This is particularly true for new apps.

.NET Framework deals with this by typically flipping AppContext flags on by default for the target framework that they are introduced in. That's effectively analogous to the behavior of having them on by default in the version they were introduced in in .NET (core).

@vatsan-madhavan
Copy link
Member

vatsan-madhavan commented Jul 11, 2023

Please take a look at this to see how AppContext flags are plumbed down into WpfGfx. You'll have to further search the codebase for where MIL_RT_IS_DISABLE_MULTIMON_DISPLAY_CLIPPING_VALID is defined to see how the overall plumbing works.

bool? enableMultiMonitorDisplayClipping =
System.Windows.CoreCompatibilityPreferences.EnableMultiMonitorDisplayClipping;
if (enableMultiMonitorDisplayClipping != null)
{
// The flag is explicitly set by the user in application manifest
command.flags |= (UInt32)MILRTInitializationFlags.MIL_RT_IS_DISABLE_MULTIMON_DISPLAY_CLIPPING_VALID;
if (!enableMultiMonitorDisplayClipping.Value)
{
command.flags |= (UInt32) MILRTInitializationFlags.MIL_RT_DISABLE_MULTIMON_DISPLAY_CLIPPING;
}
}
if (CoreAppContextSwitches.DisableDirtyRectangles)
{
command.flags |= (UInt32)MILRTInitializationFlags.MIL_RT_DISABLE_DIRTY_RECTANGLES;
}

Given that MilRTInitializationFlags has room for expansion, this is probably an apt way to communicate DX=LatestSupported vs. DX=v9 information down to MIL.

This information should be plumbed down at the very start of WpfGfx thread, and known early.

I don't understand whether you evaluated this existing mechanism and decided to do something else out of (unspecified) necessity, or whether you were unaware of this mechanism existing in WpfGfx. If the latter, please rework the changes (and if former, it would help reviewers to understand rationale).

@vatsan-madhavan
Copy link
Member

vatsan-madhavan commented Jul 11, 2023

@batzen - Yes, you're correct. While the registry solution may not be suitable for all users, we have discovered that several switches in the unmanaged code rely on this approach. Link

 

Although using AppContextSwitch might have been a better option, we need to investigate its viability in the unmanaged part of WPF.

See my previous comment about how AppContext switch plumbing works all the way into WpfGfx.

@vatsan-madhavan
Copy link
Member

System.Windows.Media.isHardwareAccelerationEnabledForRDP

In this context, isHardwareAccelerationEnabledForRDP seems fine, but I'd suggest something slighly clearer like

  • EnableHardwareAccelerationInRDP (if shipping disabled by default, and supplying an opt-in quirk) or
  • DisableHardwareAccelerationInRDP (if shipping enabled by default and supplying an opt-out quirk)

In either case, the idea is that the quirks should be structured like this:

  • false by default when TFM is current framework/runtime version
    • Can be explicitly opted-in by applications targeting the current framework/runtime version by setting it to true
  • true by default when TFM is v.Next framework/runtime version

@Kuldeep-MS
Copy link
Member Author

Please take a look at this to see how AppContext flags are plumbed down into WpfGfx. You'll have to further search the codebase for where MIL_RT_IS_DISABLE_MULTIMON_DISPLAY_CLIPPING_VALID is defined to see how the overall plumbing works.

bool? enableMultiMonitorDisplayClipping =
System.Windows.CoreCompatibilityPreferences.EnableMultiMonitorDisplayClipping;
if (enableMultiMonitorDisplayClipping != null)
{
// The flag is explicitly set by the user in application manifest
command.flags |= (UInt32)MILRTInitializationFlags.MIL_RT_IS_DISABLE_MULTIMON_DISPLAY_CLIPPING_VALID;
if (!enableMultiMonitorDisplayClipping.Value)
{
command.flags |= (UInt32) MILRTInitializationFlags.MIL_RT_DISABLE_MULTIMON_DISPLAY_CLIPPING;
}
}
if (CoreAppContextSwitches.DisableDirtyRectangles)
{
command.flags |= (UInt32)MILRTInitializationFlags.MIL_RT_DISABLE_DIRTY_RECTANGLES;
}

Given that MilRTInitializationFlags has room for expansion, this is probably an apt way to communicate DX=LatestSupported vs. DX=v9 information down to MIL.

This information should be plumbed down at the very start of WpfGfx thread, and known early.

I don't understand whether you evaluated this existing mechanism and decided to do something else out of (unspecified) necessity, or whether you were unaware of this mechanism existing in WpfGfx. If the latter, please rework the changes (and if former, it would help reviewers to understand rationale).

Certainly! To provide more detail, I noticed that the flag I am sending is similar in nature to the flag used for forced software rendering in wpfgfx. Therefore, I decided to replicate the mechanism already in place for passing down that flag to wpfgfx, and use it for my new flag as well, just like the existing flag.

@Kuldeep-MS
Copy link
Member Author

@vatsan-madhavan I tested the codepath you recommended to determine if it is appropriate for my use case. I discovered that the flag you suggested as a reference is being set every time a Hwnd is created, but my flag should only be set once throughout the entire lifecycle of the application.

I looked into various code paths to find a suitable location to introduce the logic for passing down the flag to wpfgfx, but I couldn't find a better place than the current code changes where the flag is being communicated to wpfgfx.

According to our requirements, we need to pass down the flag to wpfgfx early on and only once. After my investigation, I believe that the current code changes are the most resilient to any potential regression that may occur if I were to modify the existing code flow. However, I was unable to find any code flow that meets our requirements.

@Kuldeep-MS Kuldeep-MS merged commit d990dd6 into dotnet:main Aug 11, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Sep 10, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
PR metadata: Label to tag PRs, to facilitate with triage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable h/w acceleration in RDP
9 participants