Skip to content

Commit

Permalink
Remove the need for resetLocalAttrs. Fixes #994, #952.
Browse files Browse the repository at this point in the history
The fix was made possible by the very helpful information provided by @retronym.

This commit does two key things:
 1. changes the owner when splicing original trees into new trees
 2. ensures the synthetic trees that get spliced into original trees do not need typechecking

Given this original source (from Defaults.scala):

  ...
  lazy val sourceConfigPaths = Seq(
    ...
    unmanagedSourceDirectories := Seq(scalaSource.value, javaSource.value),
    ...
  )
  ...

After expansion of .value, this looks something like:

    unmanagedSourceDirectories := Seq(
      InputWrapper.wrapInit[File](scalaSource),
      InputWrapper.wrapInit[File](javaSource)
    )

where wrapInit is something like:

    def wrapInit[T](a: Any): T

After expansion of := we have (approximately):

    unmanagedSourceDirectories <<=
      Instance.app( (scalaSource, javaSource) ) {
        $p1: (File, File) =>
          val $q4: File = $p1._1
          val $q3: File = $p1._2
          Seq($q3, $q4)
      }

So,

 a) `scalaSource` and `javaSource` are user trees that are spliced into a tuple constructor after being temporarily held in `InputWrapper.wrapInit`
 b) the constructed tuple `(scalaSource, javaSource)` is passed as an argument to another method call (without going through a val or anything) and shouldn't need owner changing
 c) the synthetic vals $q3 and $q4 need their owner properly set to the anonymous function
 d) the references (Idents) $q3 and $q4 are spliced into the user tree `Seq(..., ...)` and their symbols need to be the Symbol for the referenced vals
 e) generally, treeCopy needs to be used when substituting Trees in order to preserve attributes, like Types and Positions

changeOwner is called on the body `Seq($q3, $q4)` with the original owner sourceConfigPaths to be changed to the new anonymous function.
In this example, no owners are actually changed, but when the body contains vals or anonymous functions, they will.

An example of the compiler crash seen when the symbol of the references is not that of the vals:

symbol value $q3 does not exist in sbt.Defaults.sourceConfigPaths$lzycompute
	at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:49)
	at scala.tools.nsc.Global.abort(Global.scala:254)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIdent$1(GenICode.scala:1038)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1044)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1246)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1244)
   ...

Other problems with the synthetic tree when it is spliced under the original tree often result in type mismatches or some other compiler error that doesn't result in a crash.

If the owner is not changed correctly on the original tree that gets spliced under a synthetic tree, one way it can crash the compiler is:

java.lang.IllegalArgumentException: Could not find proxy for val $q23: java.io.File in List(value $q23, method apply, anonymous class $anonfun$globalCore$5, value globalCore, object Defaults, package sbt, package <root>) (currentOwner= value dir )
   ...
     while compiling: /home/mark/code/sbt/main/src/main/scala/sbt/Defaults.scala
        during phase: global=lambdalift, atPhase=constructors
   ...
  last tree to typer: term $outer
              symbol: value $outer (flags: <synthetic> <paramaccessor> <triedcooking> private[this])
   symbol definition: private[this] val $outer: sbt.BuildCommon
                 tpe: <notype>
       symbol owners: value $outer -> anonymous class $anonfun$87 -> value x$298 -> method derive -> class BuildCommon$class -> package sbt
      context owners: value dir -> value globalCore -> object Defaults -> package sbt
   ...

The problem here is the difference between context owners and the proxy search chain.
  • Loading branch information
harrah authored and eed3si9n committed Mar 21, 2014
1 parent c7fc85f commit 3d9ab61
Show file tree
Hide file tree
Showing 9 changed files with 142 additions and 82 deletions.
1 change: 1 addition & 0 deletions main/settings/src/main/scala/sbt/std/InputWrapper.scala
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ object InputWrapper
sel.setPos(pos) // need to set the position on Select, because that is where the compileTimeOnly check looks
val tree = ApplyTree(TypeApply(sel, TypeTree(tpe) :: Nil), ts.tree :: Nil)
tree.setPos(ts.tree.pos)
tree.setType(tpe)
c.Expr[T](tree)
}

Expand Down
44 changes: 24 additions & 20 deletions main/settings/src/main/scala/sbt/std/TaskMacro.scala
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,10 @@ object TaskMacro
final val InputTaskCreateDynName = "createDyn"
final val InputTaskCreateFreeName = "createFree"

def taskMacroImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[T]): c.Expr[Initialize[Task[T]]] =
def taskMacroImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[T]): c.Expr[Initialize[Task[T]]] =
Instance.contImpl[T,Id](c, FullInstance, FullConvert, MixedBuilder)(Left(t), Instance.idTransform[c.type])

def taskDynMacroImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[Initialize[Task[T]]]): c.Expr[Initialize[Task[T]]] =
def taskDynMacroImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[Initialize[Task[T]]]): c.Expr[Initialize[Task[T]]] =
Instance.contImpl[T,Id](c, FullInstance, FullConvert, MixedBuilder)(Right(t), Instance.idTransform[c.type])

/** Implementation of := macro for settings. */
Expand Down Expand Up @@ -177,7 +177,7 @@ object TaskMacro
private[this] def transformMacroImpl(c: Context)(init: c.Tree)(newName: String): c.Tree =
{
import c.universe.{Apply,ApplyTag,newTermName,Select,SelectTag}
val target =
val target =
c.macroApplication match {
case Apply(Select(prefix, _), _) => prefix
case x => ContextUtil.unexpectedTree(x)
Expand Down Expand Up @@ -223,7 +223,7 @@ object TaskMacro

private[this] def inputTaskMacro0[T: c.WeakTypeTag](c: Context)(t: c.Expr[T]): c.Expr[Initialize[InputTask[T]]] =
iInitializeMacro(c)(t) { et =>
val pt = iParserMacro(c)(et) { pt =>
val pt = iParserMacro(c)(et) { pt =>
iTaskMacro(c)(pt)
}
c.universe.reify { InputTask.make(pt.splice) }
Expand Down Expand Up @@ -269,22 +269,22 @@ object TaskMacro
case _ => Converted.NotApplicable
}
val util = ContextUtil[c.type](c)
util.transformWrappers(t, (nme,tpe,tree) => expand(nme,tpe,tree))
util.transformWrappers(t, (nme,tpe,tree,original) => expand(nme,tpe,tree))
}

private[this] def iParserMacro[M[_], T](c: Context)(t: c.Expr[T])(f: c.Expr[T] => c.Expr[M[T]])(implicit tt: c.WeakTypeTag[T], mt: c.WeakTypeTag[M[T]]): c.Expr[State => Parser[M[T]]] =
{
val inner: Transform[c.type,M] = new Transform[c.type,M] { def apply(in: c.Tree): c.Tree = f(c.Expr[T](in)).tree }
Instance.contImpl[T,M](c, ParserInstance, ParserConvert, MixedBuilder)(Left(t), inner)
}

private[this] def iTaskMacro[T: c.WeakTypeTag](c: Context)(t: c.Expr[T]): c.Expr[Task[T]] =
Instance.contImpl[T,Id](c, TaskInstance, TaskConvert, MixedBuilder)(Left(t), Instance.idTransform)

private[this] def inputTaskDynMacro0[T: c.WeakTypeTag](c: Context)(t: c.Expr[Initialize[Task[T]]]): c.Expr[Initialize[InputTask[T]]] =
private[this] def inputTaskDynMacro0[T: c.WeakTypeTag](c: Context)(t: c.Expr[Initialize[Task[T]]]): c.Expr[Initialize[InputTask[T]]] =
{
import c.universe.{Apply=>ApplyTree,_}

val tag = implicitly[c.WeakTypeTag[T]]
val util = ContextUtil[c.type](c)
val it = Ident(util.singleton(InputTask))
Expand All @@ -295,9 +295,13 @@ object TaskMacro
val defs = util.collectDefs(ttree, isAnyWrapper)
val checkQual = util.checkReferences(defs, isAnyWrapper)

// the Symbol for the anonymous function passed to the appropriate Instance.map/flatMap/pure method
// this Symbol needs to be known up front so that it can be used as the owner of synthetic vals
val functionSym = util.functionSymbol(ttree.pos)
var result: Option[(Tree, Type, ValDef)] = None

def subWrapper(tpe: Type, qual: Tree): Tree =
// original is the Tree being replaced. It is needed for preserving attributes.
def subWrapper(tpe: Type, qual: Tree, original: Tree): Tree =
if(result.isDefined)
{
c.error(qual.pos, "Implementation restriction: a dynamic InputTask can only have a single input parser.")
Expand All @@ -306,9 +310,9 @@ object TaskMacro
else
{
qual.foreach(checkQual)
val vd = util.freshValDef(tpe, qual.symbol) // val $x: <tpe>
val vd = util.freshValDef(tpe, qual.symbol.pos, functionSym) // val $x: <tpe>
result = Some( (qual, tpe, vd) )
val tree = util.refVal(vd, qual.pos) // $x
val tree = util.refVal(original, vd) // $x
tree.setPos(qual.pos) // position needs to be set so that wrapKey passes the position onto the wrapper
assert(tree.tpe != null, "Null type: " + tree)
tree.setType(tpe)
Expand All @@ -335,25 +339,25 @@ object TaskMacro
taskMacroImpl[I](c)( c.Expr[I](tx) )
def wrapTag[I: WeakTypeTag]: WeakTypeTag[Initialize[Task[I]]] = weakTypeTag

def sub(name: String, tpe: Type, qual: Tree): Converted[c.type] =
def sub(name: String, tpe: Type, qual: Tree, selection: Tree): Converted[c.type] =
{
val tag = c.WeakTypeTag[T](tpe)
InitParserConvert(c)(name, qual)(tag) transform { tree =>
subWrapper(tpe, tree)
subWrapper(tpe, tree, selection)
}
}

val tx = util.transformWrappers(ttree, (n,tpe,tree) => sub(n,tpe,tree))
val tx = util.transformWrappers(ttree, (n,tpe,tree,replace) => sub(n,tpe,tree,replace))
result match {
case Some((p, tpe, param)) =>
val fCore = Function(param :: Nil, tx)
val fCore = util.createFunction(param :: Nil, tx, functionSym)
val bodyTpe = wrapTag(tag).tpe
val fTpe = util.functionType(tpe :: Nil, bodyTpe)
val fTag = c.WeakTypeTag[Any](fTpe) // don't know the actual type yet, so use Any
val fInit = c.resetLocalAttrs( expandTask(false, fCore)(fTag).tree )
val fInit = expandTask(false, fCore)(fTag).tree
inputTaskCreate(InputTaskCreateDynName, tpe, tag.tpe, p, fInit)
case None =>
val init = c.resetLocalAttrs( expandTask[T](true, tx).tree )
val init = expandTask[T](true, tx).tree
inputTaskCreateFree(tag.tpe, init)
}
}
Expand All @@ -362,10 +366,10 @@ object TaskMacro
object PlainTaskMacro
{
def task[T](t: T): Task[T] = macro taskImpl[T]
def taskImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[T]): c.Expr[Task[T]] =
def taskImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[T]): c.Expr[Task[T]] =
Instance.contImpl[T,Id](c, TaskInstance, TaskConvert, MixedBuilder)(Left(t), Instance.idTransform[c.type])

def taskDyn[T](t: Task[T]): Task[T] = macro taskDynImpl[T]
def taskDynImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[Task[T]]): c.Expr[Task[T]] =
def taskDynImpl[T: c.WeakTypeTag](c: Context)(t: c.Expr[Task[T]]): c.Expr[Task[T]] =
Instance.contImpl[T,Id](c, TaskInstance, TaskConvert, MixedBuilder)(Right(t), Instance.idTransform[c.type])
}
10 changes: 3 additions & 7 deletions main/src/main/scala/sbt/Defaults.scala
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ object Defaults extends BuildCommon
scalaOrganization :== ScalaArtifacts.Organization,
buildDependencies <<= Classpaths.constructBuildDependencies,
taskTemporaryDirectory := { val dir = IO.createTemporaryDirectory; dir.deleteOnExit(); dir },
onComplete := { val dir = taskTemporaryDirectory.value; () => IO.delete(dir); IO.createDirectory(dir) },
onComplete := { val dir = taskTemporaryDirectory.value; () => {IO.delete(dir); IO.createDirectory(dir) }},
concurrentRestrictions <<= defaultRestrictions,
parallelExecution :== true,
sbtVersion := appConfiguration.value.provider.id.version,
Expand Down Expand Up @@ -459,9 +459,7 @@ object Defaults extends BuildCommon
{
val parser = loadForParser(definedTestNames)( (s, i) => testOnlyParser(s, i getOrElse Nil) )
Def.inputTaskDyn {
val res = parser.parsed
val selected = res._1
val frameworkOptions = res._2
val (selected, frameworkOptions) = parser.parsed
val s = streams.value
val filter = testFilter.value
val config = testExecution.value
Expand Down Expand Up @@ -647,9 +645,7 @@ object Defaults extends BuildCommon
import DefaultParsers._
val parser = loadForParser(discoveredMainClasses)( (s, names) => runMainParser(s, names getOrElse Nil) )
Def.inputTask {
val res = parser.parsed
val mainClass = res._1
val args = res._2
val (mainClass, args) = parser.parsed
toError(scalaRun.value.run(mainClass, data(classpath.value), args, streams.value.log))
}
}
Expand Down
17 changes: 17 additions & 0 deletions sbt/src/sbt-test/project/setting-macro/build.sbt
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import complete.DefaultParsers._

name := {
// verify lazy vals are handled (#952)
lazy val x = "sdf"
x
}

lazy val demo = inputKey[String]("sample")

def parser: complete.Parser[(Int,String)] = token(Space ~> IntBasic <~ Space) ~ token("red")

demo := {
// verify pattern match on the lhs is handled (#994)
val (n, s) = parser.parsed
s * n
}
4 changes: 4 additions & 0 deletions sbt/src/sbt-test/project/setting-macro/test
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# this test is currently just to verify the project is loaded successfully

# ensure the project is loaded
> demo 3 red
79 changes: 53 additions & 26 deletions util/appmacro/src/main/scala/sbt/appmacro/ContextUtil.scala
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,15 @@ object ContextUtil {
/** Utility methods for macros. Several methods assume that the context's universe is a full compiler (`scala.tools.nsc.Global`).
* This is not thread safe due to the underlying Context and related data structures not being thread safe.
* Use `ContextUtil[c.type](c)` to construct. */
final class ContextUtil[C <: Context](val ctx: C)
final class ContextUtil[C <: Context](val ctx: C)
{
import ctx.universe.{Apply=>ApplyTree,_}

val powerContext = ctx.asInstanceOf[reflect.macros.runtime.Context]
val global: powerContext.universe.type = powerContext.universe
def callsiteTyper: global.analyzer.Typer = powerContext.callsiteTyper
val initialOwner: Symbol = callsiteTyper.context.owner.asInstanceOf[ctx.universe.Symbol]

lazy val alistType = ctx.typeOf[AList[KList]]
lazy val alist: Symbol = alistType.typeSymbol.companionSymbol
lazy val alistTC: Type = alistType.typeConstructor
Expand All @@ -52,20 +57,23 @@ final class ContextUtil[C <: Context](val ctx: C)
* (The current implementation uses Context.fresh, which increments*/
def freshTermName(prefix: String) = newTermName(ctx.fresh("$" + prefix))

/** Constructs a new, local ValDef with the given Type, a unique name,
* the same position as `sym`, and an empty implementation (no rhs). */
def freshValDef(tpe: Type, sym: Symbol): ValDef =
/** Constructs a new, synthetic, local ValDef Type `tpe`, a unique name,
* Position `pos`, an empty implementation (no rhs), and owned by `owner`. */
def freshValDef(tpe: Type, pos: Position, owner: Symbol): ValDef =
{
val vd = localValDef(TypeTree(tpe), EmptyTree)
vd setPos getPos(sym)
val SYNTHETIC = (1 << 21).toLong.asInstanceOf[FlagSet]
val sym = owner.newTermSymbol(freshTermName("q"), pos, SYNTHETIC)
setInfo(sym, tpe)
val vd = ValDef(sym, EmptyTree)
vd.setPos(pos)
vd
}

lazy val parameterModifiers = Modifiers(Flag.PARAM)

/** Collects all definitions in the tree for use in checkReferences.
* This excludes definitions in wrapped expressions because checkReferences won't allow nested dereferencing anyway. */
def collectDefs(tree: Tree, isWrapper: (String, Type, Tree) => Boolean): collection.Set[Symbol] =
def collectDefs(tree: Tree, isWrapper: (String, Type, Tree) => Boolean): collection.Set[Symbol] =
{
val defs = new collection.mutable.HashSet[Symbol]
// adds the symbols for all non-Ident subtrees to `defs`.
Expand Down Expand Up @@ -106,17 +114,17 @@ final class ContextUtil[C <: Context](val ctx: C)

/** Constructs a tuple value of the right TupleN type from the provided inputs.*/
def mkTuple(args: List[Tree]): Tree =
{
val global: Global = ctx.universe.asInstanceOf[Global]
global.gen.mkTuple(args.asInstanceOf[List[global.Tree]]).asInstanceOf[ctx.universe.Tree]
}

def setSymbol[Tree](t: Tree, sym: Symbol): Unit =
t.asInstanceOf[global.Tree].setSymbol(sym.asInstanceOf[global.Symbol])
def setInfo[Tree](sym: Symbol, tpe: Type): Unit =
sym.asInstanceOf[global.Symbol].setInfo(tpe.asInstanceOf[global.Type])

/** Creates a new, synthetic type variable with the specified `owner`. */
def newTypeVariable(owner: Symbol, prefix: String = "T0"): TypeSymbol =
{
val global: Global = ctx.universe.asInstanceOf[Global]
owner.asInstanceOf[global.Symbol].newSyntheticTypeParam(prefix, 0L).asInstanceOf[ctx.universe.TypeSymbol]
}

/** The type representing the type constructor `[X] X` */
lazy val idTC: Type =
{
Expand All @@ -136,21 +144,42 @@ final class ContextUtil[C <: Context](val ctx: C)
/** >: Nothing <: Any */
def emptyTypeBounds: TypeBounds = TypeBounds(definitions.NothingClass.toType, definitions.AnyClass.toType)

/** Creates a new anonymous function symbol with Position `pos`. */
def functionSymbol(pos: Position): Symbol =
callsiteTyper.context.owner.newAnonymousFunctionValue(pos.asInstanceOf[global.Position]).asInstanceOf[ctx.universe.Symbol]

def functionType(args: List[Type], result: Type): Type =
{
val global: Global = ctx.universe.asInstanceOf[Global]
val tpe = global.definitions.functionType(args.asInstanceOf[List[global.Type]], result.asInstanceOf[global.Type])
tpe.asInstanceOf[Type]
}

/** Create a Tree that references the `val` represented by `vd`. */
def refVal(vd: ValDef, pos: Position): Tree =
/** Create a Tree that references the `val` represented by `vd`, copying attributes from `replaced`. */
def refVal(replaced: Tree, vd: ValDef): Tree =
treeCopy.Ident(replaced, vd.name).setSymbol(vd.symbol)

/** Creates a Function tree using `functionSym` as the Symbol and changing `initialOwner` to `functionSym` in `body`.*/
def createFunction(params: List[ValDef], body: Tree, functionSym: Symbol): Tree =
{
val t = Ident(vd.name)
assert(vd.tpt.tpe != null, "val type is null: " + vd + ", tpt: " + vd.tpt.tpe)
t.setType(vd.tpt.tpe)
t.setPos(pos)
t
changeOwner(body, initialOwner, functionSym)
val f = Function(params, body)
setSymbol(f, functionSym)
f
}

def changeOwner(tree: Tree, prev: Symbol, next: Symbol): Unit =
new ChangeOwnerAndModuleClassTraverser(prev.asInstanceOf[global.Symbol], next.asInstanceOf[global.Symbol]).traverse(tree.asInstanceOf[global.Tree])

// Workaround copied from scala/async:can be removed once https://github.com/scala/scala/pull/3179 is merged.
private[this] class ChangeOwnerAndModuleClassTraverser(oldowner: global.Symbol, newowner: global.Symbol) extends global.ChangeOwnerTraverser(oldowner, newowner)
{
override def traverse(tree: global.Tree) {
tree match {
case _: global.DefTree => change(tree.symbol.moduleClass)
case _ =>
}
super.traverse(tree)
}
}

/** Returns the Symbol that references the statically accessible singleton `i`. */
Expand All @@ -164,7 +193,6 @@ final class ContextUtil[C <: Context](val ctx: C)

/** Returns the symbol for the non-private method named `name` for the class/module `obj`. */
def method(obj: Symbol, name: String): Symbol = {
val global: Global = ctx.universe.asInstanceOf[Global]
val ts: Type = obj.typeSignature
val m: global.Symbol = ts.asInstanceOf[global.Type].nonPrivateMember(global.newTermName(name))
m.asInstanceOf[Symbol]
Expand All @@ -176,7 +204,6 @@ final class ContextUtil[C <: Context](val ctx: C)
**/
def extractTC(tcp: AnyRef with Singleton, name: String)(implicit it: ctx.TypeTag[tcp.type]): ctx.Type =
{
val global: Global = ctx.universe.asInstanceOf[Global]
val itTpe = it.tpe.asInstanceOf[global.Type]
val m = itTpe.nonPrivateMember(global.newTypeName(name))
val tc = itTpe.memberInfo(m).asInstanceOf[ctx.universe.Type]
Expand All @@ -187,8 +214,8 @@ final class ContextUtil[C <: Context](val ctx: C)
/** Substitutes wrappers in tree `t` with the result of `subWrapper`.
* A wrapper is a Tree of the form `f[T](v)` for which isWrapper(<Tree of f>, <Underlying Type>, <qual>.target) returns true.
* Typically, `f` is a `Select` or `Ident`.
* The wrapper is replaced with the result of `subWrapper(<Type of T>, <Tree of v>)` */
def transformWrappers(t: Tree, subWrapper: (String, Type, Tree) => Converted[ctx.type]): Tree =
* The wrapper is replaced with the result of `subWrapper(<Type of T>, <Tree of v>, <wrapper Tree>)` */
def transformWrappers(t: Tree, subWrapper: (String, Type, Tree, Tree) => Converted[ctx.type]): Tree =
{
// the main tree transformer that replaces calls to InputWrapper.wrap(x) with
// plain Idents that reference the actual input value
Expand All @@ -197,7 +224,7 @@ final class ContextUtil[C <: Context](val ctx: C)
override def transform(tree: Tree): Tree =
tree match
{
case ApplyTree(TypeApply(Select(_, nme), targ :: Nil), qual :: Nil) => subWrapper(nme.decoded, targ.tpe, qual) match {
case ApplyTree(TypeApply(Select(_, nme), targ :: Nil), qual :: Nil) => subWrapper(nme.decoded, targ.tpe, qual, tree) match {
case Converted.Success(t, finalTx) => finalTx(t)
case Converted.Failure(p,m) => ctx.abort(p, m)
case _: Converted.NotApplicable[_] => super.transform(tree)
Expand Down
Loading

0 comments on commit 3d9ab61

Please sign in to comment.