-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
8338596: Clarify handling of restricted and caller-sensitive methods #21067
base: master
Are you sure you want to change the base?
Conversation
👋 Welcome back mcimadamore! A progress list of the required criteria for merging this PR into |
@mcimadamore This change is no longer ready for integration - check the PR body for details. |
@mcimadamore The following labels will be automatically applied to this pull request:
When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command. |
Webrevs
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the java/lang/foreign
package still the right place for this? (Maybe it should be under java/lang
).
Java API elements which can then be used directly by clients. For instance the | ||
restricted method <a href="../MemorySegment.html#reinterpret(long)"><code>MemorySegment.reinterpret(long)</code></a> | ||
can be used to create a fresh segment with the same address and temporal bounds, but with | ||
the provided size. This can be useful to resize memory segments obtained when | ||
interacting with native functions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This example is now talking about 'segment'/'address'/'temporal bounds' without the context of the other text in the package-info file, which seems a bit confusing. I suggest removing the example here, as essentially the same example is given in the next paragraph as well.
Some methods in the Java SE API are considered <em>restricted</em>. Restricted methods | ||
are typically used to bind native foreign data and/or functions to first-class |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like a short general description is warranted here as well. Maybe something like: 'Restricted methods are APIs that can, when used incorrectly, violate the integrity of the Java Virtual Machine, but are conditionally made available to users, as they provide essential functionality'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, the current text is still very FFM centric.
I agree, the scope for this is now outside the foreign package. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I wrote in the CSR request for the JEP:
I think each method that is restricted and/or caller-sensitive should specify what happens when called when there is no caller context. We should use
AccessibleObject::canAccess
as an exemplar here:I have no doubt other caller-sensitive methods have failed to do this to date, but that should be fixed.
This has to be mentioned in e.g. the javadoc for System.loadLibrary
.
I don't disagree that the javadoc for Then there's the question of: JNI libraries are associated with class loaders. The class loader affected by a |
Update restricted method page
<title>Restricted methods</title> | ||
</head> | ||
<body> | ||
<h1 id="restricted">Restricted methods</h1> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've kept the text more general. I took the liberty to use (and tweak) some text in here:
https://cr.openjdk.org/~iris/se/23/spec/fr/java-se-23-fr-spec/
The example is now gone - while we could still provide some example, I believe it's hard not to get too drawn into specifics. I think stating that these methods can violate integrity is probably enough.
be disabled by granting access to restricted methods to selected modules. This can be | ||
done either via implementation-specific command line options or programmatically, e.g. | ||
by calling <a href="../../ModuleLayer.Controller.html#enableNativeAccess(java.lang.Module)"><code>ModuleLayer.Controller.enableNativeAccess(java.lang.Module)</code></a>.</p> | ||
<p>When a restricted method is invoked by <a href="../../../../specs/jni/index.html">JNI code</a>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMHO, having this text here is better than copy and paste a similar version of this text in 14 (!!) places. That's why we added the @Restricted
annotation, to make sure that uniform javadoc is generated for all restricted methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At some point we may have to re-visit the word "access". All restricted methods are accessible (public methods in public classes in exported packages) so I think we'll need to more clearly distinguish this from the native or restricted operation that the methods perform. Okay for now as this PR is mostly moving text.
/csr needed |
@mcimadamore has indicated that a compatibility and specification (CSR) request is needed for this pull request. @mcimadamore please create a CSR request for issue JDK-8338596 with the correct fix version. This pull request cannot be integrated until the CSR request is approved. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
…ods.html Co-authored-by: Jorn Vernee <JornVernee@users.noreply.github.com>
And I strongly disagree because the only reason I conceded that this documentation issue need not be addressed by the CSR request for JEP 472 was because JDK-8338596 was filed to address it - ref the comment from @jddarcy : |
Just be clarify: is it your expectation that, in order for this issue to be fixed, we should fix the javadoc of all caller sensitive methods, addressing issues that have nothing to do with restricted-ness? If so, then my reading of this issue was incorrect, I will close this PR, and file a separate, more targeted, issue to improve the situation of restricted methods only (which is the goal of this PR). Separately, while I agree that the javadoc for caller sensitive methods is lacking in many ways, I don't see the connection between correctly specifying what classloader is used in |
No, just the new restricted methods. The doc for other context-sensitive methods may be lacking but that is a separate issue. I don't know what @jddarcy had in mind. |
by calling <a href="../../ModuleLayer.Controller.html#enableNativeAccess(java.lang.Module)"><code>ModuleLayer.Controller.enableNativeAccess(java.lang.Module)</code></a>.</p> | ||
<p>When a restricted method is invoked by <a href="../../../../specs/jni/index.html">JNI code</a>, | ||
or from an <a href="../Linker.html#upcallStub(java.lang.invoke.MethodHandle,java.lang.foreign.FunctionDescriptor,java.lang.foreign.Arena,java.lang.foreign.Linker.Option...)">upcall stub</a> | ||
and a Java caller can not be determined, it is as if the restricted method call occurred in an <em>unnamed module</em>.</p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any scenario where there are Java frames on the stack but calling through a native frame and back to Java with an upcall leads to the "can not be determined". I can't think of any so wonder if this can be changed to say "no caller class on the stack" as is done in the several CS methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggested the current wording here: #21067 (comment)
I think 'no caller on the stack' is too vague. AFAICT, the mechanism by which a CS method determines its caller is not documented (if it is, we should link to that here). Also, I think a user might reasonably ask: "In which cases would there not be a caller class on the stack?". So, I suggested the blanket statement instead, rather than leaning on poorly defined concepts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We use "no caller class" in the CS methods, maybe it could be improved.
I think "can not be determined" just begs questions. Is this a JDK limitation, something I'm doing wrong, or something else, ... if you see what I mean.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is documented by the CS JEP: https://openjdk.org/jeps/176
It discovers its caller's class by invoking the sun.reflect.Reflection.getCallerClass method.
CS set the precedent here and the terminology.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think "can not be determined" just begs questions. Is this a JDK limitation, something I'm doing wrong, or something else, ... if you see what I mean.
I think saying 'no caller class on the stack' has the same effect though, unless someone knows how the implementation works (which may not be unreasonable), and is aware of the scenario where a java method is called from a new native thread. I thought it would look cleaner to have this be more 'explicitly unspecified' instead. But, maybe having something (vague or not), is better than nothing...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is documented by the CS JEP: https://openjdk.org/jeps/176
I read the JEP, but it only refers to sun.reflect.Reflection.getCallerClass
, which is an internal API that I don't expect external users to know about/understand.
To be clear: I don't think we should explain exactly how CS methods determine the caller. IMO, the fact that this is implemented using a stack walk is an implementation detail that users shouldn't have to know about (and it seems plausible for VM implementations to use other mechanisms). Or at least, that's what I would like to think.
In lieu of that, I think it would be better to say that: in general (depending on the VM impl) there may be cases in which the caller can not be determined, and then as an example explicitly enumerate the cases in which the caller class can not be determined by the reference implementation. That explanation can then be linked to from the CS methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The whole notion of calls is implicitly stack-based so I'm not sure what your concern is here. A caller sensitive method's behaviour is affected by who its Java caller is, and that is implicitly "up the stack". Whether we walk the stack dynamically or have it recorded elsewhere for quick look up is not relevant, but the fact the caller is in the stack is simply a fact.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The "stack" is exposed in the API with StackWalker, Thread::getStackTrace and other APIs. For CS and restricted methods then I think we are trying to convey that there are no Java frames on the caller stack. Several existing CS APIs document the case where there is "no caller frame on the stack". This is mostly to cover the case where a CS method is invoked from a JNI attached thread. If there are native frames sandwiched between Java frames and a CS method then the intention was that these methods use the Java frame for the check, but that is hard to specify and it sounds like we have to re-visit this a bit.
For now, I think the proposed wording in this PR is okay. There are clearly some follows up needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now, I think the proposed wording in this PR is okay.
Yes, I agree what I'm suggesting is out of scope for this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as an example explicitly enumerate the cases in which the caller class can not be determined by the reference implementation
Any accessible CS method can be called directly from JNI and so have no Java caller on the stack.
The changes in this PR means that the javadoc generated text "XXX is a restricted method of the Java platform" now links to the restricted method page. That page specifies that calling the method without a caller class will be treated as if called from code in an unnamed module. Is your ask that the javadoc generated text inline this, or the equivalent by change the method description of each restricted method? I don't think this is critical for this this PR, and arguably a corner case to be upcalling from JNI attached threads. |
I admit I had missed the fact there would be some auto-generated text and links in the API docs for |
I've reverted the sentence that refers to "no caller class on the stack". As for the remaining comments, I'm not sure how to proceed. Especially stuff like:
I don't see the connection between "restricted methods" and "simple native methods". Restricted methods, as per the new javadoc text:
It is outside the scope of the javadoc text to state exactly why each restricted method is marked as such. In general, we do not provide many clarifications in any of the existing restricted methods, as the reason for "restrictedness" is rather obvious from reading the javadoc. In the case of |
@@ -42,6 +42,7 @@ public class TestRestricted extends JavadocTester { | |||
public final ToolBox tb; | |||
public static void main(String... args) throws Exception { | |||
var tester = new TestRestricted(); | |||
tester.setAutomaticCheckLinks(false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hns I had to disable this check because otherwise the test framework will attempt (and fail) to resolve ../java.base/java/lang/doc-files/RestrictedMethods.html
. Is there a better way to do this w/o disabling the link checks?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not Hannes, but to my mind, the fact that you had to disable that check to make the test pass hints that it's a real problem rather than a minor inconvenience.
Basing this URL off {@docRoot}
seems an incorrect thing to do:
doclet.Restricted.url={@docRoot}/java.base/java/lang/doc-files/RestrictedMethods.html
It will only be @docRoot
in the JDK, but not in any other code base, and certainly not in that test.
I don't know if restricted methods is a JDK-only feature, but for javadoc's sake, we should treat it like it's not. So the link should go to the same place where {@link Object}
goes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will only be
@docRoot
in the JDK, but not in any other code base, and certainly not in that test.
Note: only JDK is allowed to have restricted methods. @Restricted
is a non-exported JDK annotation. So 3rd party libraries cannot define restricted methods. So, how to render the javadoc for restricted methods is "our own" issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for missing the mention. I agree with @pavelrappo that this doesn't look quite right. It's true that it is not usually an issue as the feature will not be used outside of Java SE APIs. But it's conceivable that API docs are built for JDK/Java SE modules without including java.base
. I think the right solution would be to create the link only if java.base
/java.lang
is included in the documentation.
I don't agree. I think restricted methods should document why they are restricted if it is not patently obvious. |
I think you are arguing for an API note in each of these methods to document the rationale for why they are restricted. I don't think it needs to be done here, that is, API notes can be added separately, as needed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<sigh> These comments never got "pushed"
done either via implementation-specific command line options or programmatically, e.g. | ||
by calling <a href="../../ModuleLayer.Controller.html#enableNativeAccess(java.lang.Module)"><code>ModuleLayer.Controller.enableNativeAccess(java.lang.Module)</code></a>.</p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the very generic preceding description of "restricted methods" with no actual example, the fact this method is called enableNativeAccess
really stands out as odd. One would expect it to be enableRestrictedAccess
.
by calling <a href="../../ModuleLayer.Controller.html#enableNativeAccess(java.lang.Module)"><code>ModuleLayer.Controller.enableNativeAccess(java.lang.Module)</code></a>.</p> | ||
<p>When a restricted method is invoked by <a href="../../../../specs/jni/index.html">JNI code</a>, | ||
or from an <a href="../Linker.html#upcallStub(java.lang.invoke.MethodHandle,java.lang.foreign.FunctionDescriptor,java.lang.foreign.Arena,java.lang.foreign.Linker.Option...)">upcall stub</a> | ||
and a Java caller can not be determined, it is as if the restricted method call occurred in an <em>unnamed module</em>.</p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The whole notion of calls is implicitly stack-based so I'm not sure what your concern is here. A caller sensitive method's behaviour is affected by who its Java caller is, and that is implicitly "up the stack". Whether we walk the stack dynamically or have it recorded elsewhere for quick look up is not relevant, but the fact the caller is in the stack is simply a fact.
by calling <a href="../../ModuleLayer.Controller.html#enableNativeAccess(java.lang.Module)"><code>ModuleLayer.Controller.enableNativeAccess(java.lang.Module)</code></a>.</p> | ||
<p>When a restricted method is invoked by <a href="../../../../specs/jni/index.html">JNI code</a>, | ||
or from an <a href="../Linker.html#upcallStub(java.lang.invoke.MethodHandle,java.lang.foreign.FunctionDescriptor,java.lang.foreign.Arena,java.lang.foreign.Linker.Option...)">upcall stub</a> | ||
and a Java caller can not be determined, it is as if the restricted method call occurred in an <em>unnamed module</em>.</p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as an example explicitly enumerate the cases in which the caller class can not be determined by the reference implementation
Any accessible CS method can be called directly from JNI and so have no Java caller on the stack.
Hmm seems github got confused and some of my comments last week did already get pushed through. Probably best to clearly state where I currently stand:
Thanks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The JavaDoc parts mostly look good, but a few details about how the doc file path is created can be improved. Since I'm more familiar with the code I could contribute the changes. Alternatively, I can help with any questions.
@@ -424,7 +424,9 @@ doclet.ReflectivePreviewAPI={0} refers to one or more reflective preview APIs: | |||
doclet.UsesDeclaredUsingPreview={0} refers to one or more types which are declared using a preview feature of the Java language: {1}. | |||
doclet.PreviewTrailingNote1=Programs can only use {0} when preview features are enabled. | |||
doclet.PreviewTrailingNote2=Preview features may be removed in a future release, or upgraded to permanent features of the Java platform. | |||
doclet.RestrictedLeadingNote={0} is a restricted method of the Java platform. | |||
doclet.Restricted.url={@docRoot}/java.base/java/lang/doc-files/RestrictedMethods.html |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no reason to add the path to the properties, as it will not be internationalized. The place to put this would be jdk.javadoc.internal.doclets.toolkit.util.DocPaths
, but I would only define the file name RestrictedMethods.html
there as the rest of the path can be resolved as a doc-file relative to the java.lang
package.
@@ -42,6 +42,7 @@ public class TestRestricted extends JavadocTester { | |||
public final ToolBox tb; | |||
public static void main(String... args) throws Exception { | |||
var tester = new TestRestricted(); | |||
tester.setAutomaticCheckLinks(false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for missing the mention. I agree with @pavelrappo that this doesn't look quite right. It's true that it is not usually an issue as the feature will not be used outside of Java SE APIs. But it's conceivable that API docs are built for JDK/Java SE modules without including java.base
. I think the right solution would be to create the link only if java.base
/java.lang
is included in the documentation.
https://download.java.net/java/early_access/jdk24/docs/specs/jni/design.html#calling-caller-sensitive-methods is the documentation about the invocation of a caller-sensitive method via JNI when there is no Java caller on the stack. CS methods do not behave in the same way, for example some API throws NPE or a default behavior . We examined all CS methods and the API specification specifies the behavior JDK-8177155 when a CS method is invoked via JNI but no Java caller on the stack. I think the documentation is clear enough. |
I created a sub-PR for streamlining integration of this feature into JavaDoc code: mcimadamore#22 |
Thanks! |
Only link restricted method doc-file if it is available
/contributor add @hns |
|
@mcimadamore |
This PR moves the section on restricted methods from the the javadoc of
java.lang.foreign
package into a standalone static javadoc page.This is because, after JEP 472, we now have restricted methods outside the foreign package, namely
System::loadLibrary
,Runtime::loadLibrary
(and related methods). And, even before, we also had a restricted method inModuleLayer.Controller
.The new static page contains some guidance of what happens when a restricted method is called when there's no Java frame on the stack (this can happen e.g. when upcalling into a restricted method from a native thread not known to the JVM) - that is, the call is treated as originating from an unnamed module.
The static page is linked from the restricted method banner in a restricted method javadoc. Here's an example.
Progress
Issues
Reviewers
Contributors
<hannesw@openjdk.org>
Reviewing
Using
git
Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/21067/head:pull/21067
$ git checkout pull/21067
Update a local copy of the PR:
$ git checkout pull/21067
$ git pull https://git.openjdk.org/jdk.git pull/21067/head
Using Skara CLI tools
Checkout this PR locally:
$ git pr checkout 21067
View PR using the GUI difftool:
$ git pr show -t 21067
Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/21067.diff
Webrev
Link to Webrev Comment