Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ZEPPELIN-2865] upgrade Beam interpreter to latest version #2541

Closed
wants to merge 2 commits into from

Conversation

mingmxu
Copy link

@mingmxu mingmxu commented Aug 15, 2017

What is this PR for?

upgrade Beam interpreter to use the latest version of Apache Beam.

What type of PR is it?

[Improvement]

Todos

What is the Jira issue?

How should this be tested?

  • Start the Zeppelin server
  • The prefix of interpreter is %beam and then write your code with required imports and the runner

Refer to docs/interpreter/beam.md for an example;

Screenshots (if appropriate)

Questions:

  • Does the licenses files need update? no
  • Is there breaking changes for older versions? no
  • Does this needs documentation? yes, updated docs/interpreter/beam.md and README.md

@mingmxu mingmxu force-pushed the ZEPPELIN-2865 branch 2 times, most recently from 0a73b36 to 46ad7ef Compare August 15, 2017 22:12
@1ambda
Copy link
Member

1ambda commented Aug 16, 2017

Thanks for the contribution. Could you check the failed job?

@mingmxu
Copy link
Author

mingmxu commented Aug 16, 2017

sure, seems an error in UI test, let me trigger it again.

@@ -8,12 +8,9 @@ Current interpreter implementation supports the static repl. It compiles the cod
You have to first build the Beam interpreter by enable the **beam** profile as follows:

```
mvn clean package -Pbeam -DskipTests
mvn clean package -Pbeam -DskipTests -Pscala-2.10
Copy link
Member

@1ambda 1ambda Aug 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, you removed the ### Notice section and added -Pscala-2.10.
It's quite unclear to use scala 2.10 only or not. Can we use scala-2.11+ as of beam 2.0?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still scala-2.10 only in Beam 2.0, let me restore the notice message. --Btw, does Zeppelin provide a release version with scala-2.11(donot find in download page)? Wanna know what will happen with Beam.

Copy link
Member

@1ambda 1ambda Aug 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xumingmin Hi Zeppelin can be built with scala-2.11

You can use it like

./dev/change_scala_version.sh 2.11
mvn clean package...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @1ambda . I verified the libraries in Beam, that it only supports scala_2.10 so far.

@1ambda
Copy link
Member

1ambda commented Aug 17, 2017

LGTM. Merge if no more discussion.

@asfgit asfgit closed this in b87bcf5 Aug 20, 2017
@mingmxu mingmxu deleted the ZEPPELIN-2865 branch October 2, 2017 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants