Byzer-lang v2.3.0
发布时间 2022-04-29
很高兴今天我们发布了 Byzer（Byzer-lang） 2.3.0 版本，提供了更加灵活且稳定的 Byzer 引擎。
主要内容更新
-
在该版本中，我们优化了产品的安装部署流程，将配置文件和可执行脚本分离，并在可执行脚本中引入了环境检查，并优化了终端的信息显示。用户可通过调整
${BYZER_HOME/conf目录下的配置文件, 然后执行${BYZER_HOME/bin/byzer.sh start | stop | restart}就可以对 Byzer 引擎进行启动，停止以及重启等操作，在易用性和可维护性上得到了巨大的提升。设计详情请参考 BIP 3: Optimize Byzer lang script -
在
2.3.0版本中，我们优化了 Byzer 引擎的打包逻辑并修复了部分 bug（源代码可参见 byzer-org/byzer-build），统一了各产品包的项目结构以及使用方式，同时优化了官方下载站点的文件目录结构，将项目中历史遗留的 naming 问题做了统一，期望能够给社区用户带来更好的体验。
该变更可能会影响使用旧版本的用户，相关变更如下：
| 变更说明 | Before 2.3.0 | 2.3.0 |
|---|---|---|
| byzer 镜像内目录改名 | /home/deploy/mlsql |
/home/deploy/byzer-lang |
| byzer 镜像中主程序 jar 路径变更 | /home/deploy/mlsql/libs |
/home/deploy/byzer-lang/main |
| 主程序 jar 名称变更 | streamingpro-mlsql-spark_3.0_2.12-2.2.2.jar |
byzer-lang-3.1.1-2.12-2.3.0.jar |
| 默认删除 Spark velocity jar | spark velocity jar 可能会造成 json path 的读取问题 | 在 all-in-one 以及镜像中将该 jar 文件进行了删除 |
-
在
2.3.0版本的开发过程中，我们对官方手册 做了大量的修订和内容增补，涵盖了部署安装，REST 数据源等重点章节，期望能够给用户更多的帮助。 -
在
2.3.0版本的开发过程中，我们优化了 Byzer 引擎的 CICD 流程，增强了 Byzer 引擎的自动化测试能力，提高了代码的覆盖率，自动化测试环境覆盖了 Byzer 引擎支持的多种环境，设计详情可以参考 BIP 1: Support Integration testing using docker
优化改进
- byzer-lang/issues/1737 REST 数据源上传功能增加配置目录支持
- byzer-lang/issues/1771 动态安装插件支持设置代理
- byzer-lang/issues/1772 支持 Oracle upsert 语义，设计详情见 BIP 2: Support upserting oracle table
缺陷修复
- byzer-lang/issues/1670 修复了 REST 数据源请求中 limit 参数执行和预期不一致的问题
- byzer-lang/issues/1738 修复了 REST 数据源请求当请求失败应该返回状态码的问题
- byzer-build/issues/51 默认在产品包以及镜像中删除了 velocity jar
- byzer-build/issues/32 修复了 Byzer Build 项目中命名不明确的问题
下载使用
下载 Byzer-lang 2.3.0 安装包，请点击下方链接获取：
https://download.byzer.org/byzer/2.3.0/
安装部署指南请查看手册： Byzer引擎部署指引
反馈
如果您遇到疑问或者问题，请到 byzer-lang 项目的 Issue 里留言，
或通过邮件联系 PMC Member：pmc@byzer.org
欢迎大家积极反馈使用体验与遇到的问题，共同参与社区建设。
致谢
感谢在 Byzer-lang 2.3.0 版本中做出贡献社区小伙伴们（排名不分先后）：anan0120，kaliGo-Li，AdmondGuo，WilliamZhu，hellozepp，jiachuan.zhu，Zhengshuai PENG，Yifan Chen，Qi Lin，ckeys
Release Date: 2022-04-29
We are pleased to announce that today we officially released the Byzer-lang 2.3.0 version, which provides a more flexible and stable Byzer engine.
Major Changes
-
In this version, we have optimized the installation and deployment process of Byzer engine, separated configuration files from executable scripts, introduced environment checks, and optimized the display of terminal information. User can change the configuration by modifying the property files in
${BYZER_HOME/conf, and can start, stop, restart the Byzer engine by executing${BYZER_HOME/bin/byzer.sh start | stop | restart}. This change brings a huge improvement in ease of use and maintainability. Details please refer to the BIP 3: Optimize Byzer lang script -
In the
2.3.0version, we optimized the packaging logic of the Byzer engine and fixed some bugs (see byzer-org/byzer-build), unifies the project structure and usage of each product offer, and optimizes the file directory structure of the official download site, Fix the naming problem left over in the project, hoping to bring a better experience to community users.
This may introduce the break changes， details as following：
| Change | Before 2.3.0 | 2.3.0 |
|---|---|---|
| byzer folder name change in image | /home/deploy/mlsql |
/home/deploy/byzer-lang |
| byzer main jar path change in image | /home/deploy/mlsql/libs |
/home/deploy/byzer-lang/main |
| main jar name change | streamingpro-mlsql-spark_3.0_2.12-2.2.2.jar |
byzer-lang-3.1.1-2.12-2.3.0.jar |
| Remove Spark velocity jar | spark velocity jar in conflict with json path parsing | Remove this jar file in all-in-one package and images |
-
During the development of the
2.3.0version, we made a lot of revisions and additions to the official manual, hoping to give users more help. -
During the development of the
2.3.0version, we optimized the CICD process of the Byzer engine, enhanced the automated testing capabilities, and improved the code coverage. Details please refer to BIP 1: Support Integration testing using docker
Enhancement
- byzer-lang/issues/1737 Add path configuration on REST data source
- byzer-lang/issues/1771 Support set proxy when install plugins online
- byzer-lang/issues/1772 Support Oracle upsert details please refer to BIP 2: Support upserting oracle table
Bugfix
- byzer-lang/issues/1670 Fix the problem of rest api request where the param limit is executed more times than expected
- byzer-lang/issues/1738 Rest ET should return status code if http request failed
- byzer-build/issues/51 Delete spark velocity jar by default
- byzer-build/issues/32 Fix the naming problems in Byzer Build project
Downloads
To download the Byzer binary installation package, click the link below:
https://download.byzer.org/byzer/2.3.0/
Feedback
If you meet any problem, please submit an issue in project Byzer-lang
Or contact PMC Member：pmc@byzer.org
Welcome data developers to join and build community ecology.
Acknowledgments
Thanks to those who contributed code to Byzer-lang 2.3.0 (the following ranks are in no particular order): anan0120, kaliGo-Li, AdmondGuo, WilliamZhu, hellozepp, jiachuan.zhu, Zhengshuai PENG, Yifan Chen, Qi Lin, ckeys