-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unify treatment of built-in functions and SAMs #4971
Conversation
@@ -3215,7 +3146,7 @@ trait Typers extends Adaptations with Tags with TypersTracking with PatternTyper | |||
// less expensive than including them in inferMethodAlternative (see below). | |||
def shapeType(arg: Tree): Type = arg match { | |||
case Function(vparams, body) => | |||
functionType(vparams map (_ => AnyTpe), shapeType(body)) | |||
functionType(vparams map (_ => AnyTpe), shapeType(body)) // TODO: should this be erased when retyping during erasure? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In theory we shouldn't be doing overload resolution at that point, as we don't clear symbols of Select
-s before retyping.
However, I did spot a place where overload resolution does happen late in the compiler. In Mixin:
def staticCall(target: Symbol) = {
def implSym = implClass(sym.owner).info.member(sym.name)
assert(target ne NoSymbol,
List(sym + ":", sym.tpe, sym.owner, implClass(sym.owner), implSym,
beforePrevPhase(implSym.tpe), phase) mkString " "
)
typedPos(tree.pos)(Apply(staticRef(target), transformSuper(qual) :: args))
}
Here, the member
call can return an overloaded symbol, and we hope and pray that overload resolution under the erased types finds the right method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No wait, implSym
is only used in a assertion failure message. False alarm.
Tested with:
scala> trait T { private def foo(a: Option[Int], b: Any) = 1; private def foo(a: Option[String], b: String) = 2; def bar = foo(Some[Int](1), "") }
defined trait T
scala> new T{}.bar
res1: Int = 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- I'll remove the comment.
TODO list
@retronym suggests:
from @lrytz:
|
@retronym, could you elaborate on 6ad9b44 and how I should extend its logic to indy sammy? My guess would be that we don't need to adapt unless we're supplying a sam to java? EDIT: that's probably wrong, I'll unmelt brain tomorrow and take another stab. Hopefully this will be the last issue, then I'll work on adding test cases. |
@adriaanm If the anon-class version of the SAM function would have required a bridge method, we need to either rely on the bridge created by The logic in that commit would need to be extended to compare the impl method param types with the corresponding param type of the interface method. Currently, the knowledge the If any of these require:
... we need to emit the adapter method. We also need an adapter if the return type needs to be adapted from |
Thanks! I think that's it: I'll see about generalizing the decision about boxing of the SAM's parameters and result type. |
bf1c34a
to
e8b58ac
Compare
I have a good feeling about this last commit 🚀 |
Only when using a bootstrapped compiler:
Looking into what's different about Tuple2Zipped. |
0718f36
to
dbee5c5
Compare
I've been poking around without much luck. I suspected the logic in determining which result type to use for the bridge, but not sure. Still trying to find a good way to minimize the bytecode diff between bootstrapped (fails) and non-bootstrapped version (ok). |
9527cb2
to
1c4bd93
Compare
1c4bd93
to
f83e665
Compare
ffed567
to
a0721a2
Compare
8c57d6e
to
60ba12b
Compare
@retronym, good catch on excluding nested/local types from SAM conversion. I haven't found an example of why |
I think I had in mind a correctness problem based on my attempts to use install default methods in However, there is a performance problem:
Or, by dumping a stack trace within the lambda body (including Hidden Frames, #TIL), we see that we route through the boxing bridge
|
Ok, it seems like enough of a corner case to start by excluding them. Can always open it up again. |
Serialization seems to work (although warrants a test)
I wasn't sure whether the backend would add the |
I wonder if we can/should find a way to make the same exclusion for |
12e2886
to
fcc5fad
Compare
@adriaanm Er ... I would very much hope that the SAM treatment gets precedence over implicit conversion! What if there is no explicit type on the lambda parameter? The implicit would not be applicable, but the SAM treatment should. And that means that, if implicit takes precedence over SAM, the behavior would be different whether or not there's an explicit type on the parameter. |
When recovering missing argument types for an eta-expanded method value, rework the expected type to a method type.
Thanks, @retronym, for the super thorough review! 🔍 I think I've addressed all your feedback. I hope the last commit won't break anything -- I think it's optional for M4. I'll re-run the community build and hopefully we'll get the 👍 for M4 tomorrow! |
// type `m` directly (undoing eta-expansion of method m) to determine the argument types. | ||
val ptUnrollingEtaExpansion = | ||
if (paramsMissingType.nonEmpty && pt != ErrorType) fun.body match { | ||
case Apply(meth, args) if (vparams corresponds args) { case (p, Ident(name)) => p.name == name case _ => false } => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Took me a while to convince myself that this (existing) code was correct, usually matching on names like this is sketchy.
But there is no way that an Ident
argument to the method can bind to anything other than the eponymous function parameter.
{ val a = ""; (a, b) => bar(a /*Ident(a) can't be the outer a*/, b) }
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a note.
With the latest 5d7d644, the Scala.js base (main) test suite passes :-) |
OK the Scala.js CI (mostly) passes. The only failing tests are So this PR is green from Scala.js' point of view. |
Jason points out the recursion will be okay if type checking the function inside the eta-expansion provides fully determined argument types, as the result type is not relevant for this phase of typedFunction.
Community build is looking good! The one failure was expected.
|
s/conversions/conversion "trigger for a literal function" Perhaps a short example would be helpful here. |
// The constructor only exists if the trait's template has statements. | ||
// Sadly, we can't be more precise without access to the tree that defines the SAM's owner. | ||
!sym.primaryConstructor.exists && | ||
(sym.isInterface || sym.info.decls.forall(mem => mem.isMethod || mem.isType)) // TODO OPT: && {sym setFlag INTERFACE; true}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be .members
(now that this method doesn't recurse?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It still maps ok
over the ancestors.
LGTM |
Function literal can have user-defined SAM or built-in function type.
A function literal, such as
x => x
is now accepted when aSingle Abstract Method (SAM) type is expected, in addition to
the usual Scala function type (e.g.,
Int => Int
). A SAM typeis Java 8's convention for functions, so that e.g.,
Runnable
is now recognized as a function type.
Traditionally, Scala has shipped with FunctionN classes to
represent functions, with
Int => Int
being syntactic sugarfor
Function1[Int, Int]
. Scala 2.11 introduced experimentalsupport for expanding function literals to anonymous subclasses
of SAM type, and with Scala 2.12.0-M4, we now compile these
literals to the same bytecode as Java 8, whether targeting a
FunctionN or a user-defined functional interface.
Thanks to our new trait encoding where a trait compiles to
a Java 8 interface, our built-in FunctionN traits now also
compile to Functional Interfaces, so that Java programmers
can use Java 8's syntax for closures to target them.
If your project defines an implicit conversion from a SAM type
to a compatible Scala FunctionN type, this conversion will
no longer trigger, as the compiler performs SAM conversion
before looking for an implicit view, which enables generating
more efficient bytecode (using invokedynamic & LambdaMetaFactory).
To retain the old behavior, you may compile under
-Xsource:2.11
,or disqualify the type from being a SAM (e.g., by adding a
second abstract method).
For example,
Here, SAM conversion from the function literal
(x: Int) => x.toString
to the expected SAM type
MySam
takes precedence over insertingthe implicit view
fun2sam((x: Int) => x.toString)
, which is how the2.11 type checker expands the last line of the example.
Examples: Source Incompatible Change
Here are some snippets that compile in 2.11, but fail in 2.12. Roughly speaking, this is because SAM types and FunctionN types now compete harder during overload resolution. The Function type still wins (as explained in the spec update), but type inference breaks down when we cannot reduce the overloaded candidates down to one before type checking the arguments:
In the 1MLoC of the community build, we had less than a handful of lines that needed to change. Here's an example from akka-actor:
Implementation Details and Restrictions
Scala considers any top-level class (trait or not) with exactly
one abstract method and an accessible zero-argument a SAM type.
(See the language specification for more details.)
On the Java 8 platform, only a subset of our SAM types can be
compiled to a combination of invokedynamic and LambdaMetaFactory,
to avoid having to emit bytecode for the anymous class at
compile time. Since Java 8's LambdaMetaFactory (LMF) is not
aware of Scala's trait encoding, it can only properly
instantiate traits that correspond to "pure" Java interfaces --
that is, interfaces that do not require synthetic members to
be implemented (by the compiler) when inherited in a proper class.
This rules out nested traits, traits with fields, traits that
extend a non-trait class (except for Object), traits with any
statements in their template body (for example, LMF would
not be able to run the initializer that's required to
execute the println in
trait T { println("hello!") }
),Traits that cannot be instantiated by LMF are still SAM targets,
the compiler simply creates anonymous subclasses as before,
instead of deferring spinning them up at linkage time
(when invokedynamic's bootstrap method calls LMF).
Finally, when a SAM type is specialized (more precisely,
its specialized type parameter is instantiated with a
type for which it is specialized), we do not use
LambdaMetaFactory as incompatible with Scala specialization.
Specialization
The current specialization scheme is not amenable to using
LambdaMetaFactory to spin up subclasses. Since the generic
method is abstract, and the specialized ones are concrete,
specialization is rendered moot because we cannot implement
the specialized method with the lambda using LMF.
For the specialized versions of built-in function types,
we use hand-crafted versions, as illustrated next.
Notice how
apply$mcB$sp
is looking pretty SAMmy:Contrast this with our specialized standard FunctionN:
The single abstract method in
Function0$mcB$sp
isapply
, andthe method that would let us avoid boxing, if it were abstract,
is
apply$mcB$sp
...Type Checker Internals
A function node is first type checked, and parameter types are
inferred, regardless of whether the expected function type is one
of our built-in FunctionN classes, or a user-defined Single
Abstract Method type (argument types are derived from both kinds
of expected types).
typedFunction
always assigns a built-inFunctionN
type to thetree, though.
Next, if the expected type is a (polymorphic) SAM type, this
creates a tension between the tree's type and the expect type.
This gap is closed by the
adapt
method usinginferSamType
,which attaches a SAMFunction attachment to the Function node,
so that the back-end can target the right method & type when
emitting the invokedynamic instruction.
We may want to improve this implementation to avoid re-typechecking
function nodes to expected SAM types, as this is pretty tricky
to juggle between erasure and specialization retypechecking trees.
Tree Shape of an expanded Function literal
As with the delambdafy phase, a function literal's body is lifted
out to a method in the surrounding scope (the "target method"),
with the actual function instance created from a lightweight
subclass of the expected type (or the corresponding
invokedynamic + LMF combo).
TODO 2.0
TODO M5