Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Combine two NUnit3 Drivers #1597

Merged
merged 9 commits into from
Jan 20, 2025
Merged

Combine two NUnit3 Drivers #1597

merged 9 commits into from
Jan 20, 2025

Conversation

CharliePoole
Copy link
Member

Fixes #1596

@CharliePoole
Copy link
Member Author

This is still in progress, but comments are helpful anyway, especially about direction I'm taking.

The first two commits result in a combined driver using conditional compilation. The driver has the working name of NUnitCombinedFrameworkDriver but I hope to find a better one, possibly just NUnitFrameworkDriver since it works for both v3 and v4.

Currently the conditional code is scattered around the file, but that will change in the next commit.

@CharliePoole
Copy link
Member Author

The third commit wraps the two APIs in individual classes and implements a compile-time strategy pattern using API2009 for .NET Framework and API2018 for .NET Core.

@CharliePoole
Copy link
Member Author

Fourth commit does some refactoring and removes the two classes being replaced. Tne new driver is called NUnitFrameworkDriver, eliminating the '3' in the name.

At this point, the choice of interface is still determined at compile time. I've tried two different approaches for making the decision at runtime and neither worked. See #1596 for discussion of problems.

@CharliePoole
Copy link
Member Author

The fifth and sixth commits don't advance towards the goal, but perform refactoring to make further progress easier.

The various Api classes moved into separate files in the fifth commit.

In the sixth, I remove extraneous code from TestAgentRunner, which is based on the possibility of having multiple drivers execute in the same appdomain. This has been unused for some time since there is a one-to-one mapping across test assemblies, agents and domains.

@CharliePoole
Copy link
Member Author

CharliePoole commented Jan 16, 2025

The seventh commit finally reaches the goal of running .NET Framework tests using the 2018 API. It's a pretty big change, doubling the number of files affected and, unfortunately, making a number of unrelated changes that were needed.

A key problem was that a newly created driver didn't have an ID set in it's constructor. Rather, it was set later after construction - usually immediately after. There were various alternative ways to handle this, all a bit disruptive. I chose to modify the constructor for the driver as well as the IDriverFactory interface. This allows the constructor to complete initialization of the ID, which is a must-not-be-null property.

Remaining to do:

  • Parameterize the driver tests so that both APIs are tested under .NET Framework
  • Select the exact version cutoff for using each API. My first guess would be 3.10+ for Api2018, but I'll try a range of them.

@manfred-brands This is still WIP but please take a look. Let me know if you'd rather see this as several PRs. Now that I know it works, I can easily create a new PR for the first six commits and make the rest of it a second PR.

UPDATE: Failing CI due to .NET 7.0 not installed. For now, I made sure it's installed. I'll remove it and add .NET 9.0 in a separate issue and PR.

@CharliePoole CharliePoole changed the title WIP: Combine two NUnit3 Drivers Combine two NUnit3 Drivers Jan 16, 2025
@CharliePoole
Copy link
Member Author

The eighth commit refactors the tests in order to ensure that both APIs we support under .NET Framework are tested. An internal constructor is added to NUnitFrameworkDriver to support the tests.

The driver now switches between the 2009 and the 2018 APIs based on the version of NUnit in use. I was happily surprised to discover that the new API works for versions 3.2 and higher. Very rudimentary package tests have been added to ensure that the switching takes place correctly, but for most cases we now use the newer API. If problems arise, it would be possible to add some sort of switch (e.g. an environment variable) to force use of the older API. I don't see much point in exposing it as a command-line option.

This is now ready for review. @manfred-brands See my earlier comment about splitting it up if it seems desirable.

Copy link
Member

@manfred-brands manfred-brands left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't get through it completely. Will continue this afternoon.
The global change looks great, there are some knitpicking things.

src/TestData/NUnit3.2/NUnit3.2.csproj Outdated Show resolved Hide resolved
src/TestData/NUnit3.10/NUnit3.10.csproj Outdated Show resolved Hide resolved
src/TestData/NUnit3.0.1/NUnit3.0.1.csproj Outdated Show resolved Hide resolved
src/NUnitEngine/nunit.engine/Runners/TestEngineRunner.cs Outdated Show resolved Hide resolved
Copy link
Member

@manfred-brands manfred-brands left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more comments/questions.

Copy link
Member

@manfred-brands manfred-brands left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small item and one question about the location of the EnterContextualReflection calls.

Comment on lines +156 to +157
var type = _frameworkAssembly.ShouldNotBeNull().GetType(typeName, throwOnError: true)!;
return Activator.CreateInstance(type, args)!;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where the using (_assemblyLoadContext.ShouldNotBeNull().EnterContextualReflection())
should be called as dependencies are loaded when calling GetType or CreateInstance.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed in our tests, but it's certainly possible it may be needed in some situation. Maybe try a test that depends on a different version of TestCentric.Metadata than the one we use. It's supposed to work as a result of our creating a special context for the tests - equivalent to the NetFramework approach of using a separate AppDomain.

@CharliePoole
Copy link
Member Author

@manfred-brands Great review. I think the code is improved a lot. Sorry it was such a slog. I think I'll try to split big things into smaller PRs for review, even if they don't represent a releasable increment.

You'll need to either re-approve or merge the PR. It appears that I've lost my ability to force a merge, likely as a side effect of turning on the automatic reviewer selection feature.

@manfred-brands
Copy link
Member

You'll need to either re-approve or merge the PR

I'm happy to leave the AssemblyContext issues to later.
However, you marked some issues I raised as resolved, but I don't see a new commit.

Copy link
Member

@manfred-brands manfred-brands left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Assuming you have a pending commit with the small changes.

@CharliePoole
Copy link
Member Author

CharliePoole commented Jan 20, 2025

I'm not sure what didn't get in. With such a large set of changes, I use "resolved" to keep track of what I need to pay attention to, resolving each item after I have changed the code or made a comment that may question the suggestion or disagree with it. Anyway, I'll double-check.

UPDATE

Apparently this works better if you actually push your changes. :-)

@CharliePoole CharliePoole merged commit 9beb29c into version4 Jan 20, 2025
4 checks passed
@CharliePoole CharliePoole deleted the issue-1596 branch January 20, 2025 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants