This project is a collection of extensions for Byzer-lang. Please check every module in project for more detail.
- Python >= 3.6
- Maven >= 3.0
clone byzer-lang, then execute
mvn clean install -DskipTests -Ponline -Phive-thrift-server -Pjython-support -Pscala-2.12 -Pspark-3.0.0 -Pstreamingpro-spark-3.0.0-adaptor
then
./dev/switch.sh 2.4
mvn clean install -DskipTests -Ponline -Phive-thrift-server -Pjython-support -Pscala-2.11 -Pspark-2.4.0 -Pstreamingpro-spark-2.4.0-adaptor
You can install mlsql_plugin_tool to build module in this project.
Install command:
pip install mlsql_plugin_tool
Build shard jar comamnd:
mlsql_plugin_tool build --module_name xxxxx --spark spark243
- spark: two options are avaiable, spark243, spark311
- module_name e.g mlsql-excel, ds-hbase-2x
Once build success, the system will show message like fowllowing:
====Build success!=====
File location 0：
/Users/allwefantasy/Volumes/Samsung_T5/allwefantasy/CSDNWorkSpace/mlsqlplugins/ds-hbase-2x/target/ds-hbase-2x-2.4_2.11-0.1.0-SNAPSHOT.jar
Then you can install this plugin(jar file) in MLSQL Engine
- binlog2delta
- connect-persist
- ds-hbase-2x
- mlsql-bigdl
- mlsql-excel
- stream-persist
- mlsql-mllib