Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update GpuJsonToStructs to use the new JNI kernel when the input schema is StructType #11348

Closed
wants to merge 24 commits into from
Closed

Update GpuJsonToStructs to use the new JNI kernel when the input schema is StructType #11348

wants to merge 24 commits into from

Conversation

ttnghia
Copy link
Collaborator

@ttnghia ttnghia commented Aug 16, 2024

TBA

@ttnghia
Change reference to MapUtils into JSONUtils
bcdf3f2 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Use JSONUtils#fromJsonToStructs
32007ab 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia ttnghia added SQL part of the SQL/Dataframe plugin P0 Must have for release labels Aug 16, 2024
@ttnghia ttnghia requested a review from revans2 August 16, 2024 23:11
@ttnghia ttnghia self-assigned this Aug 16, 2024
revans2
revans2 reviewed Aug 19, 2024
View reviewed changes
Copy link
Collaborator

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@ttnghia
Further improvement for fromJsonToStructs
6819427 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Disable debug printing
98e47c8 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Fix memory leak
4f230ef 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Add allowNonNumericNumbers option
b872022 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Merge branch 'branch-24.10' into json_to_structs
043588d
@ttnghia
Merge branch 'branch-24.10' into json_to_structs
88ac7fe
@ttnghia
Fix style, and change DataType=>DType conversion
429fbde 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Fix date and time types
4f24348 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Debugging
298a18c 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Fix boolean type
9ea5840 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Xfail tests
ff0a42a 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Revert "Xfail tests"
8a82227 
This reverts commit ff0a42a.
@ttnghia
Fix unquote strings
2162966 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Remove redundant code
ac108c4 
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
@ttnghia
Use JNI kernel for array type
069e11f
@ttnghia
Revert "Use JNI kernel for array type"
82fc36f 
This reverts commit 069e11f.
@ttnghia
Check for array of array and fallback
c9ef1c8
@ttnghia
Use isNullOrEmpty from JNI
9a9c39e
@ttnghia
Cleanup
7eaba78
@ttnghia
Merge branch 'branch-24.10' into json_to_structs
4b62e11 
# Conflicts:
#	sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonReadCommon.scala
#	sql-plugin/src/main/scala/org/apache/spark/sql/rapids/GpuJsonToStructs.scala
@ttnghia
Add unquotedControlChars option
38dcf4a
@ttnghia
Merge branch 'branch-24.10' into json_to_structs
b0e8354
@ttnghia
Copy link
Collaborator Author

ttnghia commented Oct 7, 2024

Close as we've decided to implement from_json using `cudf JSON reader, thus it is no longer needed.

@ttnghia ttnghia closed this Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@revans2 revans2 revans2 left review comments

Assignees

@ttnghia ttnghia

Labels
P0 Must have for release SQL part of the SQL/Dataframe plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ttnghia @revans2