Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[0.13.2] exception during macro expansion: java.lang.NullPointerException #1107

Closed
eed3si9n opened this issue Feb 6, 2014 · 7 comments
Closed
Labels

Comments

@eed3si9n
Copy link
Member

eed3si9n commented Feb 6, 2014

steps

  1. grab sbt/sbt-appengine tag 0.6.1
  2. change to sbt.version=0.13.2-SNAPSHOT

problem

sbt-appengine> compile
[info] Updating {file:/x/sbt-appengine/}sbt-appengine...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 2 Scala sources to /x/sbt-appengine/target/scala-2.10/sbt-0.13/classes...
[error] /x/sbt-appengine/src/main/scala/AppenginePlugin.scala:162: exception during macro expansion: 
[error] java.lang.NullPointerException
[error]     at sbt.appmacro.Instance$.sbt$appmacro$Instance$$addType$1(Instance.scala:170)
[error]     at sbt.appmacro.Instance$$anonfun$sbt$appmacro$Instance$$sub$1$1.apply(Instance.scala:178)
[error]     at sbt.appmacro.Instance$$anonfun$sbt$appmacro$Instance$$sub$1$1.apply(Instance.scala:177)
[error]     at sbt.appmacro.Converted$Success.transform(Convert.scala:33)
[error]     at sbt.appmacro.Instance$.sbt$appmacro$Instance$$sub$1(Instance.scala:177)
[error]     at sbt.appmacro.Instance$$anonfun$1.apply(Instance.scala:183)
[error]     at sbt.appmacro.Instance$$anonfun$1.apply(Instance.scala:183)
[error]     at sbt.appmacro.ContextUtil$appTransformer$2$.transform(ContextUtil.scala:227)
[error]     at sbt.appmacro.ContextUtil$appTransformer$2$.transform(ContextUtil.scala:222)
[error]     at scala.reflect.internal.Trees$class.itransform(Trees.scala:1217)
[error]     at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13)
[error]     at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13)
[error]     at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2897)
[error]     at sbt.appmacro.ContextUtil$appTransformer$2$.transform(ContextUtil.scala:232)
[error]     at sbt.appmacro.ContextUtil$appTransformer$2$.transform(ContextUtil.scala:222)
[error]     at scala.reflect.api.Trees$Transformer$$anonfun$transformTrees$1.apply(Trees.scala:2900)
[error]     at scala.reflect.api.Trees$Transformer$$anonfun$transformTrees$1.apply(Trees.scala:2900)
[error]     at scala.collection.immutable.List.loop$1(List.scala:170)
[error]     at scala.collection.immutable.List.mapConserve(List.scala:186)
[error]     at scala.reflect.api.Trees$Transformer.transformTrees(Trees.scala:2900)
[error]     at scala.reflect.internal.Trees$class.itransform(Trees.scala:1219)
[error]     at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13)
[error]     at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13)
[error]     at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2897)
[error]     at sbt.appmacro.ContextUtil$appTransformer$2$.transform(ContextUtil.scala:230)
[error]     at sbt.appmacro.ContextUtil$appTransformer$2$.transform(ContextUtil.scala:222)
[error]     at scala.reflect.api.Trees$Transformer$$anonfun$transformTrees$1.apply(Trees.scala:2900)
[error]     at scala.reflect.api.Trees$Transformer$$anonfun$transformTrees$1.apply(Trees.scala:2900)
[error]     at scala.collection.immutable.List.loop$1(List.scala:170)
[error]     at scala.collection.immutable.List.mapConserve(List.scala:186)
[error]     at scala.reflect.api.Trees$Transformer.transformTrees(Trees.scala:2900)
[error]     at scala.reflect.internal.Trees$class.itransform(Trees.scala:1219)
[error]     at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13)
[error]     at scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13)
[error]     at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2897)
[error]     at sbt.appmacro.ContextUtil$appTransformer$2$.transform(ContextUtil.scala:230)
[error]     at sbt.appmacro.ContextUtil.transformWrappers(ContextUtil.scala:236)
[error]     at sbt.appmacro.Instance$.contImpl(Instance.scala:183)
[error]     at sbt.std.TaskMacro$.iInitializeMacro(TaskMacro.scala:236)
[error]     at sbt.std.TaskMacro$.inputTaskMacro0(TaskMacro.scala:225)
[error]     at sbt.std.TaskMacro$.inputTaskMacroImpl(TaskMacro.scala:220)
[error]     at sbt.std.TaskMacro$.inputTaskAssignMacroImpl(TaskMacro.scala:135)
[error]     gae.requestLogs     := AppEngine.appcfgTask("request_logs", outputFile = Some("request.log")).evaluated,
[error]                         ^
[error] one error found
[error] (compile:compile) Compilation failed
[error] Total time: 36 s, completed Feb 6, 2014 10:03:54 AM

The line in question is:

    gae.requestLogs     := AppEngine.appcfgTask("request_logs", outputFile = Some("request.log")).evaluated,

expectation

it should compile, since it does in 0.13.1.

@cunei
Copy link

cunei commented Feb 7, 2014

The issue appeared in the nightly 0.13.1-20131123-062316. There was only one commit on that day in the 0.13 branch, likely the cause of the regression:
3b213e5

cunei referenced this issue Feb 7, 2014
The fix was made possible by the very helpful information provided by @retronym.

This commit does two key things:
 1. changes the owner when splicing original trees into new trees
 2. ensures the synthetic trees that get spliced into original trees do not need typechecking

Given this original source (from Defaults.scala):

  ...
  lazy val sourceConfigPaths = Seq(
    ...
    unmanagedSourceDirectories := Seq(scalaSource.value, javaSource.value),
    ...
  )
  ...

After expansion of .value, this looks something like:

    unmanagedSourceDirectories := Seq(
      InputWrapper.wrapInit[File](scalaSource),
      InputWrapper.wrapInit[File](javaSource)
    )

where wrapInit is something like:

    def wrapInit[T](a: Any): T

After expansion of := we have (approximately):

    unmanagedSourceDirectories <<=
      Instance.app( (scalaSource, javaSource) ) {
        $p1: (File, File) =>
          val $q4: File = $p1._1
          val $q3: File = $p1._2
          Seq($q3, $q4)
      }

So,

 a) `scalaSource` and `javaSource` are user trees that are spliced into a tuple constructor after being temporarily held in `InputWrapper.wrapInit`
 b) the constructed tuple `(scalaSource, javaSource)` is passed as an argument to another method call (without going through a val or anything) and shouldn't need owner changing
 c) the synthetic vals $q3 and $q4 need their owner properly set to the anonymous function
 d) the references (Idents) $q3 and $q4 are spliced into the user tree `Seq(..., ...)` and their symbols need to be the Symbol for the referenced vals
 e) generally, treeCopy needs to be used when substituting Trees in order to preserve attributes, like Types and Positions

changeOwner is called on the body `Seq($q3, $q4)` with the original owner sourceConfigPaths to be changed to the new anonymous function.
In this example, no owners are actually changed, but when the body contains vals or anonymous functions, they will.

An example of the compiler crash seen when the symbol of the references is not that of the vals:

symbol value $q3 does not exist in sbt.Defaults.sourceConfigPaths$lzycompute
	at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:49)
	at scala.tools.nsc.Global.abort(Global.scala:254)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIdent$1(GenICode.scala:1038)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1044)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1246)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1244)
   ...

Other problems with the synthetic tree when it is spliced under the original tree often result in type mismatches or some other compiler error that doesn't result in a crash.

If the owner is not changed correctly on the original tree that gets spliced under a synthetic tree, one way it can crash the compiler is:

java.lang.IllegalArgumentException: Could not find proxy for val $q23: java.io.File in List(value $q23, method apply, anonymous class $anonfun$globalCore$5, value globalCore, object Defaults, package sbt, package <root>) (currentOwner= value dir )
   ...
     while compiling: /home/mark/code/sbt/main/src/main/scala/sbt/Defaults.scala
        during phase: global=lambdalift, atPhase=constructors
   ...
  last tree to typer: term $outer
              symbol: value $outer (flags: <synthetic> <paramaccessor> <triedcooking> private[this])
   symbol definition: private[this] val $outer: sbt.BuildCommon
                 tpe: <notype>
       symbol owners: value $outer -> anonymous class $anonfun$87 -> value x$298 -> method derive -> class BuildCommon$class -> package sbt
      context owners: value dir -> value globalCore -> object Defaults -> package sbt
   ...

The problem here is the difference between context owners and the proxy search chain.
@retronym
Copy link
Member

retronym commented Feb 7, 2014

Let me know if you need a hand with this one @cunei; I fixed a similar problem for Mark recently.

retronym added a commit to retronym/sbt that referenced this issue Feb 7, 2014
Instead, we just take the position of the tree itself.
@retronym
Copy link
Member

retronym commented Feb 7, 2014

@cunei I'm pretty sure this will fix it. Would appreciate help to add an associate test case.

https://github.com/retronym/xsbt/compare/ticket;1107?expand=1

@retronym
Copy link
Member

retronym commented Feb 7, 2014

I just checked, this does build sbt-appengine.

Here's a minimal repro:

scala> import sbt._, Keys._
import sbt._
import Keys._

scala> def appcfgTask(a: String, b: String) = Def.task("")
appcfgTask: (a: String, b: String)sbt.Def.Initialize[sbt.Task[String]]

scala> TaskKey[Unit]("test") := appcfgTask(b = "", a = "").value
<console>:15: error: exception during macro expansion:
java.lang.NullPointerException

// After

scala> TaskKey[Unit]("test") := appcfgTask(b = "", a = "").value
res3: sbt.Def.Setting[sbt.Task[Unit]] = setting(ScopedKey(Scope(This,This,This,This),test)) at LinePosition(($iw) <console>,15)

@jsuereth jsuereth added the Bug label Feb 11, 2014
@jsuereth
Copy link
Member

@retronym is that commit good to go? I can make a PR if so...

@retronym
Copy link
Member

The code is some of the best I've ever written.

If you could turn my comment about into a test and submit the PR, I'd be eternally grateful!

@jsuereth
Copy link
Member

Sure thing. I'll leave this open in my browser, but you may need to ping me in the AM to remind me. I'm about to turn into a Father today (less literally than you).

retronym added a commit to retronym/sbt that referenced this issue Mar 10, 2014
We shouldn't assume that the qualifier of a `Select` is a
`SymTree`; it may be a `Block`. One place that happens
is after the transformation of named/defaults applications.
That causes the reported `NullPointerException'.

In any case, using `qual.symbol.pos` sense here; it yields the
position of the defintions *referred to* by `qual`, not the
position of `qual` itself.

Both problems are easily fixed: use `qual.pos` instead.

Fixes sbt#1107
eed3si9n pushed a commit to eed3si9n/sbt that referenced this issue Mar 21, 2014
We shouldn't assume that the qualifier of a `Select` is a
`SymTree`; it may be a `Block`. One place that happens
is after the transformation of named/defaults applications.
That causes the reported `NullPointerException'.

In any case, using `qual.symbol.pos` sense here; it yields the
position of the defintions *referred to* by `qual`, not the
position of `qual` itself.

Both problems are easily fixed: use `qual.pos` instead.

Fixes sbt#1107
jvican pushed a commit to scalacenter/sbt that referenced this issue May 23, 2017
We shouldn't assume that the qualifier of a `Select` is a
`SymTree`; it may be a `Block`. One place that happens
is after the transformation of named/defaults applications.
That causes the reported `NullPointerException'.

In any case, using `qual.symbol.pos` sense here; it yields the
position of the defintions *referred to* by `qual`, not the
position of `qual` itself.

Both problems are easily fixed: use `qual.pos` instead.

Fixes sbt#1107
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 4, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

4 participants