[GLUTEN-7028][CH][Part-11] Support write parquet files with bucket #8052
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Run Gluten Clickhouse CI on x86 |
11 tasks
baibaichen
changed the title
lwz9103
force-pushed
the
parquet_bucket
branch
from
0e2853a to
c692942
Compare
|
Run Gluten Clickhouse CI on x86 |
baibaichen
force-pushed
the
parquet_bucket
branch
from
c692942 to
b0e482e
Compare
|
Run Gluten Clickhouse CI on x86 |
lwz9103
force-pushed
the
parquet_bucket
branch
from
b0e482e to
c093a62
Compare
|
Run Gluten Clickhouse CI on x86 |
1 similar comment
|
Run Gluten Clickhouse CI on x86 |
lwz9103
force-pushed
the
parquet_bucket
branch
from
71bbdf3 to
47e5ffa
Compare
|
Run Gluten Clickhouse CI on x86 |
baibaichen
reviewed
Dec 3, 2024
| @@ -52,14 +52,15 @@ DB::ProcessorPtr make_sink( | |||
| const DB::Block & input_header, | |||
| const DB::Block & output_header, | |||
| const std::string & base_path, | |||
| const FileNameGenerator & generator, | |||
| FileNameGenerator & generator, | |||
baibaichen
reviewed
Dec 3, 2024
| const bool pattern; | ||
| const std::string filename_or_pattern; | ||
| // Align with org.apache.spark.sql.execution.FileNamePlaceHolder | ||
| const std::vector<std::string> placeholders = {"{id}", "{bucket}"}; |
baibaichen
reviewed
Dec 3, 2024
|
|
||
| std::string generate() const | ||
| std::string pattern_format(std::string arg, std::string replacement) const |
lwz9103
force-pushed
the
parquet_bucket
branch
from
47e5ffa to
c7f9ea0
Compare
|
Run Gluten Clickhouse CI on x86 |
baibaichen
approved these changes
Dec 3, 2024
baibaichen
added a commit
to Kyligence/gluten
that referenced
this pull request
Dec 4, 2024
baibaichen
added a commit
that referenced
this pull request
Dec 4, 2024
* [GLUTEN-1632][CH]Daily Update Clickhouse Version (20241204) * Fix Build due to ClickHouse/ClickHouse#72715 * Fix Build due to ClickHouse/ClickHouse#65691 * Fix Build due to ClickHouse/ClickHouse#72722 * Fix gtest due to #8052 * Fix benchmark due to ClickHouse/ClickHouse#72460 * Add SPARK_DIR_NAME for fixing unstable ut --------- Co-authored-by: kyligence-git <gluten@kyligence.io> Co-authored-by: Chang Chen <baibaichen@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Support write parquet files with bucket
(Fixes: #7028)
How was this patch tested?
unit tests