36 Pull requests merged by 19 people

• Core: Fix RewriteTablePath Incremental Replication

  #12172 merged Feb 8, 2025

• Auth Manager API part 5: SigV4 Auth Manager

  #11995 merged Feb 7, 2025

• Spark 3.4: Remove use of File.Separator in RewriteTablePath

  #12173 merged Feb 7, 2025

• Core, Spark: Exclude non live content file in RewriteTablePathUtil

  #12006 merged Feb 7, 2025

• Docs: Minor improvements to Spark Procedures docs

  #12190 merged Feb 7, 2025

• Update LICENSE/NOTICE for spark-runtime 3.3 and 3.4

  #12189 merged Feb 6, 2025

• Update LICENSE/NOTICE in flink-runtime jar files

  #12188 merged Feb 6, 2025

• Fix NOTICE and LICENSE in the flink-runtime jar

  #12145 merged Feb 6, 2025

• Auth Manager API part 4: RESTClient, HTTPClient

  #11992 merged Feb 6, 2025

• Fix NOTICE and LICENSE in the gcp-bundle jar

  #12144 merged Feb 6, 2025

• Fix NOTICE and LICENSE in the spark-runtime jar

  #12160 merged Feb 6, 2025

• Test Hive: Fix TestHiveMetastore

  #12140 merged Feb 6, 2025

• Remove jitpack.yml

  #12170 merged Feb 6, 2025

• API: Null check for auto-unboxed field id in builder

  #12165 merged Feb 6, 2025

• Bump Nessie to include updated License/Notice

  #12186 merged Feb 6, 2025

• Bump the versions of site/mkdocs.yml

  #12180 merged Feb 5, 2025

• Bump the versions of site/mkdocs.yml

  #12175 merged Feb 5, 2025

• Flink: Add null check to writers to prevent resurrecting null values

  #12049 merged Feb 5, 2025

• Core: Refactor variants to enable moving interfaces to API module

  #12167 merged Feb 4, 2025

• Variants: Implement toString

  #12138 merged Feb 4, 2025

• Docs: Fix latest and nightly link on javadoc (according to site README.md)

  #12023 merged Feb 4, 2025

• Spark: Test metadata tables with format-version=3

  #12135 merged Feb 4, 2025

• Spark: Update benchmark instructions

  #12171 merged Feb 4, 2025

• Core: Remove TableMetadata::Builder::resetMainBranch

  #12149 merged Feb 4, 2025

• Spark: make delete file ratio configurable

  #12148 merged Feb 3, 2025

• Build: Bump com.azure:azure-sdk-bom from 1.2.30 to 1.2.31

  #12154 merged Feb 2, 2025

• Build: Bump me.champeau.jmh:jmh-gradle-plugin from 0.7.2 to 0.7.3

  #12152 merged Feb 2, 2025

• Build: Bump net.snowflake:snowflake-jdbc from 3.21.0 to 3.22.0

  #12155 merged Feb 2, 2025

• Build: Bump software.amazon.awssdk:bom from 2.30.6 to 2.30.11

  #12156 merged Feb 2, 2025

• Build: Bump mkdocs-material from 9.5.50 to 9.6.1

  #12157 merged Feb 2, 2025

• Build: Bump nessie from 0.102.2 to 0.102.4

  #12153 merged Feb 2, 2025

• Data: open file using stats in scan

  #12151 merged Feb 2, 2025

• Spec: Fix current-version-id in View Spec example

  #12146 merged Feb 1, 2025

• Core, API, Spec: Metadata Row Lineage

  #11948 merged Feb 1, 2025

• Spark 3.5: Remove use of File.Separator in RewriteTablePath

  #12066 merged Feb 1, 2025

• Spark 3.5: Iceberg / DataFusion Comet integration

  #12147 merged Feb 1, 2025

17 Pull requests opened by 13 people

• Retry on NoSuchNamespaceException not found in rename table for rest catalog

  #12159 opened Feb 2, 2025

• spec: Remove `source-ids` for `V{1,2}` tables

  #12161 opened Feb 3, 2025

• Spec additions for encryption

  #12162 opened Feb 3, 2025

• Update documentation / add missing Iceberg table read properties

  #12163 opened Feb 3, 2025

• WIP File format write

  #12164 opened Feb 3, 2025

• Remove deprecated `WRITE_METADATA_LOCATION` and `WRITE_FOLDER_STORAGE_LOCATION`

  #12174 opened Feb 4, 2025

• Spark 3.5: Add Comet tests

  #12176 opened Feb 5, 2025

• [WIP] Ignore UnknownType in General Parquet Writer

  #12177 opened Feb 5, 2025

• Spark: DVs + Positional Deletes + Compaction

  #12181 opened Feb 5, 2025

• Docs: Deprecate data_file.distinct_counts in v3

  #12182 opened Feb 5, 2025

• Docs: Remove data_file.distinct_counts

  #12183 opened Feb 5, 2025

• Run dependabot daily

  #12184 opened Feb 5, 2025

• support source watermark for flink sql windows

  #12191 opened Feb 7, 2025

• REST: Extended header support for RESTClient implementations

  #12194 opened Feb 7, 2025

• Fix LICENSE and NOTICE for the kafka-connect-runtime distribution

  #12195 opened Feb 7, 2025

• Auth Manager API part 6: API enablement

  #12197 opened Feb 7, 2025

• support create table like in flink catalog

  #12199 opened Feb 7, 2025

15 Issues closed by 5 people

• Class TestHiveMetastore use getSystemClassLoader instead of getClass.getClassLoader in setupMetastoreDB

  #12131 closed Feb 6, 2025

• Azure Databricks Azure databricks 14.3 LTS runtime with apache iceberg 1.5.2 spark 3.5.0 and scala 2.12 Error

  #10789 closed Feb 6, 2025

• Error configuration key in document

  #10785 closed Feb 6, 2025

• Should be null_value_counts updated after adding a new column to the schema?

  #10773 closed Feb 6, 2025

• Remove orphan files without creating listed files dataframe in spark ?

  #12158 closed Feb 5, 2025

• On droping table with shared location the data got deleted which should not be case after version 0.14.0 of Apache iceberg

  #10779 closed Feb 5, 2025

• Formal verification discovers potential consistency issue

  #10720 closed Feb 5, 2025

• Make DELETE_RATIO_THRESHOLD configurable in SizeBasedDataRewriter

  #12081 closed Feb 4, 2025

• Will there be any problems migrating a hive table with 3 million partitions to Iceberg

  #10768 closed Feb 4, 2025

• Support for Default Values

  #10761 closed Feb 4, 2025

• uppercase table name not supported

  #10758 closed Feb 4, 2025

• MergeSchema doesn't work if missing columns is used for Write Ordering.

  #10751 closed Feb 4, 2025

• `add_files` procedure allows importing NULL on NOT NULL columns

  #10742 closed Feb 4, 2025

• Add geometry type to iceberg

  #2586 closed Feb 3, 2025

• Detecting duplicates in the Flink Data Stream API

  #10683 closed Feb 3, 2025

12 Issues opened by 12 people

• Iceberg SDK failed to clean up files when table has multiple references with different retention time

  #12200 opened Feb 8, 2025

• Add Zordering to table specification

  #12198 opened Feb 7, 2025

• [REST Catalog] OAuth 2 grant type "refresh_token" not implemented

  #12196 opened Feb 7, 2025

• Supports Tencent COS Object Store

  #12193 opened Feb 7, 2025

• Flink supoort of REST catalog like Apache Polaris

  #12192 opened Feb 7, 2025

• Spark streaming (merge into) iceberg table concurrent write with compaction job

  #12187 opened Feb 6, 2025

• GlueCatalog name validation

  #12185 opened Feb 5, 2025

• Question about pyspark with iceberg-aws without using the bundle jar

  #12179 opened Feb 5, 2025

• Incorrect Results and SIGSEGV on Read with Iceberg + PySpark + Nessie

  #12178 opened Feb 5, 2025

• PartitionStatsUtil#computeStats returns incomplete stats in case of partition evolution

  #12168 opened Feb 4, 2025

• Java doc link is not working

  #12166 opened Feb 3, 2025

• question on iceberg table

  #12150 opened Feb 1, 2025

61 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Data: Add partition stats writer and reader

  #11216 commented on Feb 7, 2025 • 44 new comments

• Implementation of version metadata table for view

  #12014 commented on Feb 5, 2025 • 24 new comments

• GCP: Add Iceberg Catalog for GCP BigQuery Metastore

  #11039 commented on Feb 5, 2025 • 23 new comments

• Spec: Support geo type

  #10981 commented on Feb 7, 2025 • 21 new comments

• Core: add variant type support

  #11831 commented on Feb 7, 2025 • 18 new comments

• Docs: Add rewrite-table-path in spark procedure

  #12115 commented on Feb 7, 2025 • 18 new comments

• Parquet: Implement Variant readers

  #12139 commented on Feb 5, 2025 • 16 new comments

• Spec: Update partition stats for V3

  #12098 commented on Feb 6, 2025 • 14 new comments

• Core: Add KLL Datasketch and Hive ColumnStatisticsObj as standard blo…

  #8202 commented on Feb 7, 2025 • 8 new comments

• Spark: Support singular form of years, months, days, and hours functions

  #12117 commented on Feb 4, 2025 • 5 new comments

• Range distribution iceberg sink

  #12071 commented on Feb 7, 2025 • 4 new comments

• Core: add variant builder implementation

  #11857 commented on Feb 3, 2025 • 3 new comments

• Spark: support statistics files in RewriteTablePath

  #11929 commented on Feb 8, 2025 • 2 new comments

• Core: Add InternalData read and write builders

  #12060 commented on Feb 7, 2025 • 2 new comments

• support create table like in flink catalog and watermark in windows

  #12116 commented on Feb 8, 2025 • 2 new comments

• Core, Spark: Refactor FileRewriter interface to separate planning and execution

  #11513 commented on Feb 4, 2025 • 2 new comments

• Materialized View Spec

  #11041 commented on Feb 6, 2025 • 1 new comment

• Docs: add apache amoro(incubating) with iceberg (#11965)

  #11966 commented on Feb 7, 2025 • 1 new comment

• Core: BugFix: PartitionStatsUtil#computeStats returns incomplete stats in case of partition evolution

  #12137 commented on Feb 5, 2025 • 1 new comment

• Core/RewriteFiles: Duplicate Data Bug - Fixed dropping delete files that are still required

  #10962 commented on Feb 4, 2025 • 1 new comment

• Hive: Add Hive 4 support and remove Hive runtime

  #11750 commented on Feb 6, 2025 • 0 new comments

• Open-API: Fix compilation errors in generated Java classes due to mismatched return types

  #11806 commented on Feb 6, 2025 • 0 new comments

• Reduce code duplication in VectorizedParquetDefinitionLevelReader

  #11661 commented on Feb 5, 2025 • 0 new comments

• Flink: Replace use of deprecated methods

  #11658 commented on Feb 2, 2025 • 0 new comments

• backport #11301(rowconverter) to Flink 1.19 and 1.18

  #11826 commented on Feb 7, 2025 • 0 new comments

• Doc:Hive 4.0 and later versions allow vectorized read and write opera…

  #11877 commented on Feb 3, 2025 • 0 new comments

• Adding new rewrite manifest spark action to accept custom partition o…

  #11881 commented on Feb 3, 2025 • 0 new comments

• Introduce `MissingRequiredFilesToDeleteException` for Streaming Deletes

  #11887 commented on Feb 3, 2025 • 0 new comments

• OpenAPI: Add RemoveSchemas REST update type

  #12022 commented on Feb 6, 2025 • 0 new comments

• Core: Change RemoveSnapshots to remove unused schemas

  #12089 commented on Feb 6, 2025 • 0 new comments

• Kafka Connect: Add kerberos authentication option

  #12119 commented on Feb 4, 2025 • 0 new comments

• Spark 3.5: Fix RewriteDataFiles with partial progress enabled and max-failed-commits larger than total-file-group

  #12120 commented on Feb 4, 2025 • 0 new comments

• Spark: Remove closing of IO in SerializableTable*

  #12129 commented on Feb 3, 2025 • 0 new comments

• Core: Fix cleanup of orphaned statistics files in dropTableData

  #12132 commented on Feb 4, 2025 • 0 new comments

• throw exception : InvalidOperationException(message:The following columns have types incompatible with the existing columns in their respective positions : idd1) when add column

  #3747 commented on Feb 2, 2025 • 0 new comments

• Athena Iceberg does not delete orphan files

  #10878 commented on Feb 2, 2025 • 0 new comments

• MERGE INTO TABLE is not supported temporarily.

  #10882 commented on Feb 2, 2025 • 0 new comments

• Define behavior of gc.enabled and location ownership

  #4159 commented on Feb 3, 2025 • 0 new comments

• Field comments are not written for timestamp field

  #4212 commented on Feb 3, 2025 • 0 new comments

• Spark: Add read/write support for UUIDs from bytes

  #10635 commented on Feb 3, 2025 • 0 new comments

• Namespace names with dot(.) not supported by JDBC catalog when listing namespaces

  #11990 commented on Feb 3, 2025 • 0 new comments

• REST Catalog does not validate "to" identifier on rename table

  #11154 commented on Feb 3, 2025 • 0 new comments

• Variant Data Type Support

  #10392 commented on Feb 3, 2025 • 0 new comments

• Apache Flink not committing new snapshots to Iceberg Table

  #9089 commented on Feb 4, 2025 • 0 new comments

• Some schema updates do not support dots inside a field name

  #10875 commented on Feb 4, 2025 • 0 new comments

• MERGE INTO requires sorting in already sorted iceberg tables

  #10891 commented on Feb 4, 2025 • 0 new comments

• Rest catalog: write.metadata.delete-after-commit set true not deleting expired metadata files

  #10894 commented on Feb 4, 2025 • 0 new comments

• write.wap.enabled / spark.wap.branch / spark.wap.id behavior isn't really documented

  #11528 commented on Feb 4, 2025 • 0 new comments

• Kafka Connect Sporadic Commit Delay

  #11796 commented on Feb 4, 2025 • 0 new comments

• java.lang.IllegalStateException: Connection pool shut down when close FileAppender.

  #12114 commented on Feb 4, 2025 • 0 new comments

• Do not override finalize

  #10901 commented on Feb 5, 2025 • 0 new comments

• The ColumnarToRow Spark optimization is not applied when using nested fields from an Iceberg table

  #10828 commented on Feb 5, 2025 • 0 new comments

• software.amazon.awssdk.services.s3.model.S3Exception: The bucket you are attempting to access must be addressed using the specified endpoint.

  #11997 commented on Feb 7, 2025 • 0 new comments

• zorder does not work with sub fields

  #10017 commented on Feb 7, 2025 • 0 new comments

• Parquet, Arrow: Refactor vectorized reader

  #9772 commented on Feb 6, 2025 • 0 new comments

• Kafka Connect: Add table to topics mapping property

  #10422 commented on Feb 7, 2025 • 0 new comments

• Support changelog scan for table with delete files

  #10935 commented on Feb 7, 2025 • 0 new comments

• Core: Add list/map block sizes

  #10973 commented on Feb 4, 2025 • 0 new comments

• Core: Try create Iceberg metadata table for Jdbc catalog in initialization

  #11427 commented on Feb 7, 2025 • 0 new comments

• Spark: Relativize in-memory paths for data file and rewritable delete file locations

  #11525 commented on Feb 8, 2025 • 0 new comments

• Spark 3.5: Refactor scanning changelog table with timestamps

  #11612 commented on Feb 3, 2025 • 0 new comments