Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encoder derivation for collection incompatible implementations for interpreted serde e.g. Seq instead of Vector #804

Open
chris-twiner opened this issue Mar 20, 2024 · 1 comment

Comments

@chris-twiner
Copy link
Contributor

Collection encoder uses MapObjects, this does not have special logic for Vector and will return a Seq that can be a different class.

Scope is serialisation with interpretation (many use-cases will do codegen), compilation of MapObjects is correct and uses builders. Eval of MapObjects with NewInstance fails as NewInstance will look for a Seq constructor parameter but only a Vector parameter exists.

In fixing #803 the use of Vector in UdfTests."one argument udf" failed as:

Message: scala.collection.immutable.Stream$Cons cannot be cast to scala.collection.immutable.Vector

which is true. The Stream$Cons is created by MapObjects:

  private lazy val mapElements: scala.collection.Seq[_] => Any = customCollectionCls match {
    case Some(cls) if classOf[WrappedArray[_]].isAssignableFrom(cls) =>
      ...
    case Some(cls) if classOf[scala.collection.Seq[_]].isAssignableFrom(cls) =>
      // Scala sequence
      executeFuncOnCollection(_).toSeq

There are a number of these potential issues with all similar Seq derived types (Set derived as well).

The inbuilt ScalaReflection createIterableEncoder also correctly uses the appropriate builder.

chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
@chris-twiner
Copy link
Contributor Author

This test proves it's eval only, swap forceInterpreted to forceCodeGen and it'll run.

chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
(cherry picked from commit caed43956187a52f1a37d8192b70bd9ecf297a1f)
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
@chris-twiner chris-twiner changed the title Encoder derivation for collection incompatible implementations e.g. Seq instead of Vector Encoder derivation for collection incompatible implementations for interpreted serde e.g. Seq instead of Vector Mar 20, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
…13 forced changes, compilation issue with toSeq():GenSeq
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
…13 forced changes, compilation issue with toSeq():GenSeq
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
…13 forced changes, compilation issue with toSeq():GenSeq
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
…13 forced changes, compilation issue with toSeq():GenSeq
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
…13 forced changes, compilation issue with toSeq():GenSeq
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 20, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 21, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 21, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 21, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 21, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 21, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 21, 2024
chris-twiner added a commit to chris-twiner/frameless that referenced this issue Mar 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant