Describe the use-case for this feature

Method detail...
When opening up a java/scala file containing a test class, a BSP client only knows that the class itself can be run as a test but ideally it would know what individual methods can be run.

e.g. in Java

import org.junit.jupiter.api.Test;

public class TestClass {
    @Test
    public void testFoo() { 
        // some code
    }
    @Test
    public void testBar() { 
        // some code
    }
}

In responding to buildTarget/jvmTestEnvironment, the BSP server will supply "TestClass" in its list of test mainClasses but will provide no information about testFoo or testBar. If a client such as Metals wants to let the user run these individual methods it has to be aware of the various test frameworks out there (junit, scalatest, munit etc.) to know how to query the file to identify test methods.

I think that the client shouldn't have to know any details about the test framework being used and the BSP server should supply the test methods as well as test classes.

Filtering...
Along with returning test methods, it would be probably improve performance if the buildTarget/jvmTestEnvironment request could include some kind of filtering. At the moment it returns all test classes for a list of build targets. If a user creates/changes a file/class then the BSP client has to re-request the test classes for the entire build target rather than restrict that to the document just created/changed.

What do you propose

Method detail...
Currently JvmEnvironmentItem is returned when the BSP server responds to buildTarget/jvmRunEnvironment and to buildTarget/jvmTestEnvironment.

JvmEnvironmentItem contains an array of JvmMainClass which contains a className.

Proposed options...

1. The BSP Server could simply treat className as though it was classAndMethodName and return TestClass#testFoo, TestClass#testBar. This is not very clear to the anyone implementing/using the API but could be just a change to documentation.

2. JvmMainClass could be extended to contain methodNames? : string[].
   This wouldn't make much sense for a buildTarget/jvmRunEnvironment request but it could be left blank or could be always main.

3. methods?: JvmMethods[] could be added to JvmEnvironmentItem where JvmMethod would contain a classname and an array of method names.
   Again this wouldn't make much sense in response to buildTarget/jvmRunEnvironment. It would also be partially duplicating the JvmMainClass contents.

The above options would have minimal effect on current BSP clients.

4. Create a JvmTestEnvironmentItem class and use that as a response to buildTarget/jvmTestEnvironment. Unfortunately, unlike other parts of the BSP API, the data is not set here using data and dataKind so it would probably mean that an entirely new buildTarget/jvmTestEnvironment2 would have to be created which would be painful for current users.

One thing I'm not sure on with all the above proposals: is the method name alone enough to distinguish different test methods? I'm not familiar with many test frameworks but JUnit allows parameterization of test methods so someone could theoretically define tests as...

    @ParameterizedTest
    void testFoo(int param1) {}
    @ParameterizedTest
    void testFoo(int param1, int param2) {}

or just have additional non test methods of the same name...

    void testFoo(int i) {}
    @Test
    void testFoo() {
        testFoo(1);
        testFoo(2);
    }

So would the method's signature have to be supplied as well as its name to distinguish the 2 methods in the above cases?

Currently I'm favouring proposal 2. If method signatures are needed then methodNames?: string could become methods?: JvmMethod.

Filtering...
If test methods were added to the spec then another thing that would be nice to have is a way to filter what classes should be returned.

If the test file has been changed and saved, the BSP client could requery the server for test classes/methods purely for that file/class.

Proposed change...

1. Add textDocument?: TextDocumentIdentifier[] to JvmTestEnvironmentParams so the BSP server can internally query only the required documents for new or changed test methods.

2. Add classNames?: string[] to JvmTestEnvironmentParams so the BSP server can internally query only the required documents for new or changed test methods.

3. Allow both 1) and 2) types of filtering by using data/dataKind to filter and then also hopefully future proof for different types of filtering.

I favour proposal 2. I think proposal 1 would be ideal for BSP clients but I don't think test frameworks generally allow filtering by file. Instead they filter by package/class/method.
I also can't see why a BSP client would need anything as fine as Method name filtering to discover new tests. I would have thought fully qualified classname is granular enough

I've checked to make sure there isn't already a way to support this in the current protocol

• I've checked the existing protocol and there isn't a way to do this currently

Maintainer approval (This is for the maintainers)

• Jetbrains
• Metals
• Scala Center