Closed
Description
It has been proposed to build Spark 3 support -- at least for Spark 3.2 -- against both Scala 2.12 and 2.13.
The code in spark/v3.2 currently does not compile against Scala 2.13.
Some of the sources of source incompatibility are:
scala.Predef.conforms
(deprecated since 2.11) has been removed in 2.13.scala.Seq[+A]
in 2.13 is an alias forscala.collection.immutable.Seq[A]
instead ofscala.collection.Seq[A]
. Some Spark APIs that Iceberg call takescala.Seq
parameters, instead of parameters explicitly declared asscala.collection.Seq
. (The problem is that the APIs are called from Java code, not Scala code. Java lacks the type alias, so the Java code currently usescala.collection.Seq
, but that cannot be used in 2.13 where ascala.Seq
is required.) Another manifestation of this issue isscala.collection.mutable.Buffer
cannot be used where ascala.Seq
is required.scala.collection.JavaConversions
has been removed in 2.13 (it is used in some tests).
Incidentally, scala.collection.JavaConverters
is still in 2.13 but is deprecated and use of scala.jdk.CollectionConverters
is recommended instead (but that is only in 2.13, not in 2.12).
Metadata
Metadata
Assignees
Labels
No labels
Activity