Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Private def with existential type causes spurious recompilation due to name instability #823

Closed
gkossakowski opened this issue Jul 19, 2013 · 5 comments
Assignees
Labels
Milestone

Comments

@gkossakowski
Copy link
Contributor

Consider the following code structure:

// Box.scala
class Box[T]

// Foo.scala
class Foo {
    def foo: Box[_] = null
}

// Bar.scala
class Bar {
    def f: Foo = null //we introduce dependency on Foo
}

Let's assume it's been compiled.

Now, let's introduce a private method to Foo so it looks like this:

class Foo {
    private def abc: Box[_] = null
    def foo: Box[_] = null
}

If we recompile we would expect one iteration of compilation (just to recompile Foo.scala) because the newly introduced method is private. At the moment, incremental compiler will decide to recompile Bar.scala as well because it sees changed signature of foo method. The reason its seen as changed is name instability for existential types introduced with widlcard syntax. What happens is that Scala compiler has a global counter for ids per compilation unit and uses that counter for naming existential types. Therefore even private definition can affect a signature a public definition thus causing spurious recompilation.

@ghost ghost assigned gkossakowski Jul 19, 2013
@harrah
Copy link
Member

harrah commented Jul 20, 2013

Probably due to 864580a. The correct approach was removed because it relied on Symbol.id, which is apparently an unreliable implementation detail. The id didn't need to be stable across runs, but was just used to identify equal type parameters. I don't remember the details, but should it be desirable to restore the indexed approach, perhaps a Symbol map would work instead.

@gkossakowski
Copy link
Contributor Author

Thanks for the pointer! Yes, using Symbol.id would solve that particular problem but would introduce a lot more other problems.

I was thinking about introducing some kind of a map on Sbt side. I'm not sure if Map[Symbol, Symbol] is the best idea because it means we need to allocate new symbols. Allocating symbols is a side-effecting operation (e.g. it mutates the owner) so it's dangerous to do.

I was thinking about introducing a much more specialized map Map[Symbol, String] which would just deal with remapping names of symbols corresponding to existential type variables. However, synthesizing strings that are both stable and guarantee no collisions is rather difficult.

@paulp
Copy link
Contributor

paulp commented Jul 28, 2013

Not that everything isn't a side effecting operation, but in general owners know nothing of their children. It doesn't mutate the owner unless you mutate the owner (perhaps entering it into the owner's scope.)

gkossakowski added a commit to gkossakowski/sbt that referenced this issue Aug 2, 2013
Fix the problem with unstable names synthesized for existential
types (declared with underscore syntax) by renaming type variables
to a scheme that is guaranteed to be stable no matter where given
the existential type appears.

We can do that because type variables declared in existential types
are scoped to those types so the renaming operation is rather local.

There's a specs2 unit test covering instability of existential types.
The test is included in compiler-interface project and the build
definition has been modified to enable building and executing tests
in compiler-interface project. Some dependencies has been modified:

  * compiler-interface project depends on api project for testing
    (test makes us of SameAPI)
  * dependency on junit has been introduced because it's needed
    for `@RunWith` annotation which declares that specs2 unit
    test should be ran with JUnitRunner

SameAPI has been modified to expose a method that allows us to
compare two definitions.

Also, a integration test has been added which tests scenario
mentioned in sbt#823.

This commit fixes sbt#823.
gkossakowski added a commit to gkossakowski/sbt that referenced this issue Aug 22, 2013
Fix the problem with unstable names synthesized for existential
types (declared with underscore syntax) by renaming type variables
to a scheme that is guaranteed to be stable no matter where given
the existential type appears.

We can do that because type variables declared in existential types
are scoped to those types so the renaming operation is rather local.

There's a specs2 unit test covering instability of existential types.
The test is included in compiler-interface project and the build
definition has been modified to enable building and executing tests
in compiler-interface project. Some dependencies has been modified:

  * compiler-interface project depends on api project for testing
    (test makes us of SameAPI)
  * dependency on junit has been introduced because it's needed
    for `@RunWith` annotation which declares that specs2 unit
    test should be ran with JUnitRunner

SameAPI has been modified to expose a method that allows us to
compare two definitions.

Also, a integration test has been added which tests scenario
mentioned in sbt#823.

This commit fixes sbt#823.
@peter-fu
Copy link

+1

I got to this pull request from https://gist.github.com/jroper/6374383. I think this is the pull request of the "Scala incremental compiler improvements"? The result of the fix looks amazing!

Thanks!

gkossakowski added a commit to gkossakowski/sbt that referenced this issue Oct 24, 2013
Fix the problem with unstable names synthesized for existential
types (declared with underscore syntax) by renaming type variables
to a scheme that is guaranteed to be stable no matter where given
the existential type appears.

The sheme we use are De Bruijn-like indices that capture both position
of type variable declarion within single existential type and nesting
level of nested existential type. This way we properly support nested
existential types by avoiding name clashes.

In general, we can perform renamings like that because type variables
declared in existential types are scoped to those types so the renaming
operation is local.

There's a specs2 unit test covering instability of existential types.
The test is included in compiler-interface project and the build
definition has been modified to enable building and executing tests
in compiler-interface project. Some dependencies has been modified:

  * compiler-interface project depends on api project for testing
    (test makes us of SameAPI)
  * dependency on junit has been introduced because it's needed
    for `@RunWith` annotation which declares that specs2 unit
    test should be ran with JUnitRunner

SameAPI has been modified to expose a method that allows us to
compare two definitions.

Also, a integration test has been added which tests scenario
mentioned in sbt#823.

This commit fixes sbt#823.

Proper fix for unstable existential names.
gkossakowski added a commit to gkossakowski/sbt that referenced this issue Oct 29, 2013
Fix the problem with unstable names synthesized for existential
types (declared with underscore syntax) by renaming type variables
to a scheme that is guaranteed to be stable no matter where given
the existential type appears.

The sheme we use are De Bruijn-like indices that capture both position
of type variable declarion within single existential type and nesting
level of nested existential type. This way we properly support nested
existential types by avoiding name clashes.

In general, we can perform renamings like that because type variables
declared in existential types are scoped to those types so the renaming
operation is local.

There's a specs2 unit test covering instability of existential types.
The test is included in compiler-interface project and the build
definition has been modified to enable building and executing tests
in compiler-interface project. Some dependencies has been modified:

  * compiler-interface project depends on api project for testing
    (test makes us of SameAPI)
  * dependency on junit has been introduced because it's needed
    for `@RunWith` annotation which declares that specs2 unit
    test should be ran with JUnitRunner

SameAPI has been modified to expose a method that allows us to
compare two definitions.

This commit also adds `ScalaCompilerForUnitTesting` class that allows
to compile a piece of Scala code and inspect information recorded
callbacks defined in  `AnalysisCallback` interface. That class uses
existing ConsoleLogger for logging. I considered doing the same for
ConsoleReporter. There's LoggingReporter defined which would fit our
usecase but it's defined in compile subproject that compiler-interface
doesn't depend on so we roll our own.

Also, a integration test has been added which tests scenario
mentioned in sbt#823.

This commit fixes sbt#823.
@gkossakowski gkossakowski reopened this Nov 7, 2013
@gkossakowski
Copy link
Contributor Author

I accidentally reopened this issue but clicking the wrong button on issue list. I'm very sorry for that. Closing again.

gkossakowski added a commit that referenced this issue Mar 21, 2014
Fix the problem with unstable names synthesized for existential
types (declared with underscore syntax) by renaming type variables
to a scheme that is guaranteed to be stable no matter where given
the existential type appears.

The sheme we use are De Bruijn-like indices that capture both position
of type variable declarion within single existential type and nesting
level of nested existential type. This way we properly support nested
existential types by avoiding name clashes.

In general, we can perform renamings like that because type variables
declared in existential types are scoped to those types so the renaming
operation is local.

There's a specs2 unit test covering instability of existential types.
The test is included in compiler-interface project and the build
definition has been modified to enable building and executing tests
in compiler-interface project. Some dependencies has been modified:

  * compiler-interface project depends on api project for testing
    (test makes us of SameAPI)
  * dependency on junit has been introduced because it's needed
    for `@RunWith` annotation which declares that specs2 unit
    test should be ran with JUnitRunner

SameAPI has been modified to expose a method that allows us to
compare two definitions.

This commit also adds `ScalaCompilerForUnitTesting` class that allows
to compile a piece of Scala code and inspect information recorded
callbacks defined in  `AnalysisCallback` interface. That class uses
existing ConsoleLogger for logging. I considered doing the same for
ConsoleReporter. There's LoggingReporter defined which would fit our
usecase but it's defined in compile subproject that compiler-interface
doesn't depend on so we roll our own.

ScalaCompilerForUnit testing uses TestCallback from compiler-interface
subproject for recording information passed to callbacks. In order
to be able to access TestCallback from compiler-interface
subproject I had to tweak dependencies between interface and
compiler-interface so test classes from the former are visible in the
latter. I also modified the TestCallback itself to accumulate apis in
a HashMap instead of a buffer of tuples for easier lookup.

An integration test has been added which tests scenario
mentioned in #823.

This commit fixes #823.

Conflicts:
	project/Util.scala
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants