Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump LLVM to 765206e050453018e861637a08a4520f29238074 #2835

Merged
merged 10 commits into from
Jun 13, 2024
Merged

Bump LLVM to 765206e050453018e861637a08a4520f29238074 #2835

merged 10 commits into from
Jun 13, 2024

Conversation

Connor-XY
Copy link
Contributor

This PR bumps LLVM to 765206e050453018e861637a08a4520f29238074 and stablehlo to 3c6a0673ee69e3ef0917b640ddd2ccc801e38a5c.

@AlexandreEichenberger
Copy link
Collaborator

@jenkins-droid test this please

@AlexandreEichenberger
Copy link
Collaborator

@Connor-XY thanks for pumping to the latest. Please ping me on slack/email if there are issues that you need help with.

@philass
Copy link
Member

philass commented Jun 7, 2024

@jenkins-droid test this please

@AlexandreEichenberger
Copy link
Collaborator

There are some errors, linked with deprecated casts. See the log for more info.

@Connor-XY
Copy link
Contributor Author

It seems that for s390x, the generated mlir is different from what is expected for these two test cases: conversion/krnl_to_llvm/constants_to_file/big_endian/constants.mlir and conversion/krnl_to_llvm/symbol-postfix.mlir, where llvm.call @omMMapBinaryFile cannot be found. Do you possibly know what the generated mlir is on s390x? I also found there is a small linkage issue in stablehlo. I have raised a PR and need to wait it to be merged, and then I will do a new round of upgrade.

@Connor-XY Connor-XY mentioned this pull request Jun 7, 2024
Merged
@AlexandreEichenberger
Copy link
Collaborator

@Connor-XY Thanks for looking into fixing the link error that shows up in the Linux AMD build (release mode is more fussy about libs).
Will have someone look into the s390 conversion error.

@AlexandreEichenberger
Copy link
Collaborator

@tungld there is an endianness error in the s390 build.

FAIL: Open Neural Network Frontend :: conversion/krnl_to_llvm/constants_to_file/big_endian/constants.mlir (104 of 329)
******************** TEST 'Open Neural Network Frontend :: conversion/krnl_to_llvm/constants_to_file/big_endian/constants.mlir' FAILED ********************
Exit Code: 1

hopefully a small fix. Do you mind looking into this?

@tungld
Copy link
Collaborator

tungld commented Jun 11, 2024 •
edited
Loading

Hi @Connor-XY, this patch is to fix constants.mlir:

diff --git a/test/mlir/conversion/krnl_to_llvm/constants_to_file/big_endian/constants.mlir b/test/mlir/conversion/krnl_to_llvm/constants_to_file/big_endian/constants.mlir
index 9bab5a97..82bf2fb5 100644
--- a/test/mlir/conversion/krnl_to_llvm/constants_to_file/big_endian/constants.mlir
+++ b/test/mlir/conversion/krnl_to_llvm/constants_to_file/big_endian/constants.mlir
@@ -65,10 +65,9 @@ func.func @test_constants_to_file() -> memref<10xi64> {
 // CHECK-DAG:       [[VAR_2_3_:%.+]] = llvm.mlir.constant(0 : i64) : i64
 // CHECK-DAG:       [[VAR_3_3_:%.+]] = llvm.mlir.addressof @om_external_constant_data_constant_1 : !llvm.ptr
 // CHECK-DAG:       [[VAR_4_3_:%.+]] = llvm.mlir.constant(4176 : i64) : i64
-// CHECK-DAG:       [[VAR_5_3_:%.+]] = llvm.mlir.constant(1 : i64) : i64
 // CHECK-DAG:       [[VAR_6_3_:%.+]] = llvm.mlir.addressof @om_external_constant_packedConst : !llvm.ptr
 // CHECK-DAG:       [[VAR_7_3_:%.+]] = llvm.mlir.addressof @om_external_constant_filename : !llvm.ptr
-// CHECK:           llvm.call @omMMapBinaryFile([[VAR_6_3_]], [[VAR_7_3_]], [[VAR_4_3_]], [[VAR_5_3_]]) : (!llvm.ptr, !llvm.ptr, i64, i64) -> ()
+// CHECK:           llvm.call @omMMapBinaryFile([[VAR_6_3_]], [[VAR_7_3_]], [[VAR_4_3_]], [[VAR_2_3_]]) : (!llvm.ptr, !llvm.ptr, i64, i64) -> ()
 // CHECK:           llvm.call @omGetExternalConstantAddr([[VAR_3_3_]], [[VAR_6_3_]], [[VAR_2_3_]]) : (!llvm.ptr, !llvm.ptr, i64) -> ()
 // CHECK:           llvm.call @omGetExternalConstantAddr([[VAR_1_3_]], [[VAR_6_3_]], [[VAR_0_9_]]) : (!llvm.ptr, !llvm.ptr, i64) -> ()
 // CHECK:           llvm.return

@tungld
Copy link
Collaborator

tungld commented Jun 11, 2024

and this is a patch for symbol-postfix.mlir:

diff --git a/test/mlir/conversion/krnl_to_llvm/symbol-postfix.mlir b/test/mlir/conversion/krnl_to_llvm/symbol-postfix.mlir                                                                                              [15/1945]
index 53c3591c..b684e411 100644
--- a/test/mlir/conversion/krnl_to_llvm/symbol-postfix.mlir
+++ b/test/mlir/conversion/krnl_to_llvm/symbol-postfix.mlir
@@ -62,6 +62,7 @@ module attributes {"onnx-mlir.symbol-postfix" = "tag_symbols"} {
 // CHECK:         }

 // CHECK:         llvm.func @omInputSignature_tag_symbols([[arg0_:%.+]]: !llvm.ptr) -> !llvm.ptr {
+// CHECK-DAG:       [[VAR_0_6_:%.+]] = llvm.mlir.zero : !llvm.ptr
 // CHECK-DAG:       [[VAR_0_7_:%.+]] = llvm.mlir.addressof @_entry_point_1_in_sig_tag_symbols : !llvm.ptr
 // CHECK-DAG:       [[VAR_1_5_:%.+]] = llvm.mlir.constant(27 : i64) : i64
 // CHECK-DAG:       [[VAR_2_5_:%.+]] = llvm.mlir.addressof @_entry_point_1_tag_symbols : !llvm.ptr
@@ -72,15 +73,12 @@ module attributes {"onnx-mlir.symbol-postfix" = "tag_symbols"} {
 // CHECK:           [[VAR_7_1_:%.+]] = llvm.call @strncmp([[arg0_]], [[VAR_6_3_]], [[VAR_4_3_]]) : (!llvm.ptr, !llvm.ptr, i64) -> i32
 // CHECK:           [[VAR_8_1_:%.+]] = llvm.icmp "eq" [[VAR_7_1_]], [[VAR_5_4_]] : i32
 // CHECK:           llvm.cond_br [[VAR_8_1_]], ^bb1([[VAR_3_5_]] : !llvm.ptr), ^bb2
-// CHECK:         ^bb1([[VAR_9_:%.+]]: !llvm.ptr):  // 2 preds: ^bb0, ^bb2
+// CHECK:         ^bb1([[VAR_9_:%.+]]: !llvm.ptr):  // 3 preds: ^bb0, ^bb2, ^bb2
 // CHECK:           llvm.return [[VAR_9_]] : !llvm.ptr
 // CHECK:         ^bb2:  // pred: ^bb0
 // CHECK:           [[VAR_10_1_:%.+]] = llvm.call @strncmp([[arg0_]], [[VAR_2_5_]], [[VAR_1_5_]]) : (!llvm.ptr, !llvm.ptr, i64) -> i32
 // CHECK:           [[VAR_11_1_:%.+]] = llvm.icmp "eq" [[VAR_10_1_]], [[VAR_5_4_]] : i32
-// CHECK:           llvm.cond_br [[VAR_11_1_]], ^bb1([[VAR_0_7_]] : !llvm.ptr), ^bb3
-// CHECK:         ^bb3:  // pred: ^bb2
-// CHECK:           [[VAR_12_1_:%.+]] = llvm.mlir.zero : !llvm.ptr
-// CHECK:           llvm.return [[VAR_12_1_]] : !llvm.ptr
+// CHECK:           llvm.cond_br [[VAR_11_1_]], ^bb1([[VAR_0_7_]] : !llvm.ptr), ^bb1([[VAR_0_6_]] : !llvm.ptr)
 // CHECK:         }

 // CHECK:         llvm.func @omInputSignature([[arg0_:%.+]]: !llvm.ptr) -> !llvm.ptr {
@@ -89,6 +87,7 @@ module attributes {"onnx-mlir.symbol-postfix" = "tag_symbols"} {
 // CHECK:         }

 // CHECK:         llvm.func @omOutputSignature_tag_symbols([[arg0_]]: !llvm.ptr) -> !llvm.ptr {
+// CHECK-DAG:       [[VAR_0_8_:%.+]] = llvm.mlir.zero : !llvm.ptr
 // CHECK-DAG:       [[VAR_0_9_:%.+]] = llvm.mlir.addressof @_entry_point_1_out_sig_tag_symbols : !llvm.ptr
 // CHECK-DAG:       [[VAR_1_6_:%.+]] = llvm.mlir.constant(27 : i64) : i64
 // CHECK-DAG:       [[VAR_2_6_:%.+]] = llvm.mlir.addressof @_entry_point_1_tag_symbols : !llvm.ptr
@@ -99,15 +98,12 @@ module attributes {"onnx-mlir.symbol-postfix" = "tag_symbols"} {
 // CHECK:           [[VAR_7_2_:%.+]] = llvm.call @strncmp([[arg0_]], [[VAR_6_4_]], [[VAR_4_4_]]) : (!llvm.ptr, !llvm.ptr, i64) -> i32
 // CHECK:           [[VAR_8_2_:%.+]] = llvm.icmp "eq" [[VAR_7_2_]], [[VAR_5_5_]] : i32
 // CHECK:           llvm.cond_br [[VAR_8_2_]], ^bb1([[VAR_3_6_]] : !llvm.ptr), ^bb2
-// CHECK:         ^bb1([[VAR_9_]]: !llvm.ptr):  // 2 preds: ^bb0, ^bb2
+// CHECK:         ^bb1([[VAR_9_]]: !llvm.ptr):  // 3 preds: ^bb0, ^bb2, ^bb2
 // CHECK:           llvm.return [[VAR_9_]] : !llvm.ptr
 // CHECK:         ^bb2:  // pred: ^bb0
 // CHECK:           [[VAR_10_2_:%.+]] = llvm.call @strncmp([[arg0_]], [[VAR_2_6_]], [[VAR_1_6_]]) : (!llvm.ptr, !llvm.ptr, i64) -> i32
 // CHECK:           [[VAR_11_2_:%.+]] = llvm.icmp "eq" [[VAR_10_2_]], [[VAR_5_5_]] : i32
-// CHECK:           llvm.cond_br [[VAR_11_2_]], ^bb1([[VAR_0_9_]] : !llvm.ptr), ^bb3
-// CHECK:         ^bb3:  // pred: ^bb2
-// CHECK:           [[VAR_12_2_:%.+]] = llvm.mlir.zero : !llvm.ptr
-// CHECK:           llvm.return [[VAR_12_2_]] : !llvm.ptr
+// CHECK:           llvm.cond_br [[VAR_11_2_]], ^bb1([[VAR_0_9_]] : !llvm.ptr), ^bb1([[VAR_0_8_]] : !llvm.ptr)
 // CHECK:         }
// CHECK:         llvm.func @omOutputSignature([[arg0_:%.+]]: !llvm.ptr) -> !llvm.ptr {
 // CHECK:           [[VAR_0_9_:%.+]] = llvm.call @omOutputSignature_tag_symbols([[arg0_]]) : (!llvm.ptr) -> !llvm.ptr
@@ -154,10 +150,9 @@ module attributes {"onnx-mlir.symbol-postfix" = "tag_constants_to_file"} {
 // CHECK-CONST-TO-FILE-DAG:       [[VAR_2_9_:%.+]] = llvm.mlir.constant(0 : i64) : i64
 // CHECK-CONST-TO-FILE-DAG:       [[VAR_3_9_:%.+]] = llvm.mlir.addressof @om_external_constant_data_constant_1_tag_constants_to_file : !llvm.ptr
 // CHECK-CONST-TO-FILE-DAG:       [[VAR_4_7_:%.+]] = llvm.mlir.constant(4176 : i64) : i64
-// CHECK-CONST-TO-FILE-DAG:       [[VAR_5_8_:%.+]] = llvm.mlir.constant(1 : i64) : i64
 // CHECK-CONST-TO-FILE-DAG:       [[VAR_6_6_:%.+]] = llvm.mlir.addressof @om_external_constant_packedConst_tag_constants_to_file : !llvm.ptr
 // CHECK-CONST-TO-FILE-DAG:       [[VAR_7_4_:%.+]] = llvm.mlir.addressof @om_external_constant_filename_tag_constants_to_file : !llvm.ptr
-// CHECK-CONST-TO-FILE:           llvm.call @omMMapBinaryFile([[VAR_6_6_]], [[VAR_7_4_]], [[VAR_4_7_]], {{.*}}) : (!llvm.ptr, !llvm.ptr, i64, i64) -> ()
+// CHECK-CONST-TO-FILE:           llvm.call @omMMapBinaryFile([[VAR_6_6_]], [[VAR_7_4_]], [[VAR_2_9_]], {{.*}}) : (!llvm.ptr, !llvm.ptr, i64, i64) -> ()
 // CHECK-CONST-TO-FILE:           llvm.call @omGetExternalConstantAddr([[VAR_3_9_]], [[VAR_6_6_]], [[VAR_2_9_]]) : (!llvm.ptr, !llvm.ptr, i64) -> ()
 // CHECK-CONST-TO-FILE:           llvm.call @omGetExternalConstantAddr([[VAR_1_9_]], [[VAR_6_6_]], [[VAR_0_18_]]) : (!llvm.ptr, !llvm.ptr, i64) -> ()
 // CHECK-CONST-TO-FILE:           llvm.return

Connor-XY added 10 commits June 12, 2024 21:22
@Connor-XY
bump LLVM to 765206 and stablehlo to 3c6a06
be78e6a 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
fix deprecated
575c39d 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
fix NNPA td
d0a4438 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
fix format and krnl_to_llvm test
fbb161b 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
update stablehlo
db0546d 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
fix test and stablehlo build
6d2b3cd 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
split large/small endian for symbol-postfix
e83aad5 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
fix big-endian
325007b 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
add stablehlo linalg transforms lib
e7d1f6d 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
fix merge deprecated
095d608 
Signed-off-by: Yan Xu <yan.xu0210@bytedance.com>
@Connor-XY
Copy link
Contributor Author

@tungld Would you mind having a look at symbol-postfix.milr? I found there should be 2 preds instead of 3?

@tungld
Copy link
Collaborator

tungld commented Jun 13, 2024

@tungld Would you mind having a look at symbol-postfix.milr? I found there should be 2 preds instead of 3?

Hi Yan Xu, just looked at it. It's good, the logic is unchanged. Thanks!

tungld
tungld approved these changes Jun 13, 2024
View reviewed changes
Copy link
Collaborator

@tungld tungld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. It's a huge update. Really appreciated it. Thanks!

@AlexandreEichenberger AlexandreEichenberger merged commit 0831202 into onnx:main Jun 13, 2024
7 checks passed
@AlexandreEichenberger
Copy link
Collaborator

Thanks to all for this upgrade, much appreciated.

@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #14983 [push] Bump LLVM to 765206e0504... started at 09:22

@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #14013 [push] Bump LLVM to 765206e0504... started at 10:31

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #14988 [push] Bump LLVM to 765206e0504... started at 10:22

@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #14983 [push] Bump LLVM to 765206e0504... passed after 2 hr 1 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #14988 [push] Bump LLVM to 765206e0504... passed after 2 hr 46 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #14013 [push] Bump LLVM to 765206e0504... failed after 2 hr 48 min

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tungld tungld tungld approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Connor-XY @AlexandreEichenberger @philass @tungld @jenkins-droid