hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From leesf <>
Subject [ANNOUNCE] Hudi Community Update(2021-07-18 ~ 2021-08-01)
Date Sun, 01 Aug 2021 15:49:00 GMT
Dear community,

Nice to share Hudi community bi-weekly updates for 2021-07-18 ~ 2021-08-01
with updates on features, bug fixes and tests.


[Core] Adding support to disable meta columns with bulk insert operation [1]
DeltaStreamer [2]
[Spark Integration] MergeInto Support Partial Update For COW [3]
[Hive Integration] DeltaStreamer kafka source supports consuming from
specified timestamp [4]
[Hive Integration] Adding support for HMS for running DDL queries in
hive-sync [5]
[Docs] Automate the generation of configs webpage as configs are added to
Hudi repo [6]
[Core] Adding virtual key support to COW table [7]
[Flink Integration] Add rateLimiter when Flink writes to hudi [8]
[Core] Integrate consumers with rocksDB and compression within External
Spillable Map [9]
[Flink Integration] Add option 'hive_sync.mode' for flink writer [10]
[Spark Integration] Explicit parallelism for flink bulk insert [11]
[Hive Integration] Support setting hive sync partition extractor class
based on flink configuration [12]



[Flink Integration] Remove state in BootstrapFunction [1]
[Flink Integration] Create new bucket when NewFileAssignState filled[2]
[Flink Integration] Clean and reset the bootstrap events for coordinator
when task failover [3]
[Code Cleanup] Clean up Multiple versions of scala libraries detected
Warning [4]
[Flink Integraion] Add marker files for flink writer [5]
[Spark Integration] Sync Hive Failed When Execute CTAS In Spark2 And Spark3
[Core] Fix checkpoint blocked because getLastPendingInstant() action after
than restoreWriteMetadata() action [7]
[Flink Integration] Rollback inflight compaction for flink writer [8]
[Spark Integration] MergeInto MOR Table May Result InCorrect Result [9]
[Spark Integration] Missing PrimaryKey In Hoodie Properties For CTAS Table
[Core] residual temporary files after clustering are not cleaned up [11]
[Core] Fix NPE of HoodieConfig [12]
[Core] Fix no value present in incremental query on MOR [13]
[Spark Integration] Fix Alter Partitioned Table Failed [14]
[Flink Integration] Only sync hive meta on successful commit for flink
batch writer [15]
[Core] Make codahale times transient to avoid serializable exceptions [16]
[Core]] BucketAssigner generates the fileId evenly to avoid data skew [17]
[Hive Integration] Fix database alreadyExists exception while hive sync [18]
[Spark Integration] Performance loss with the additional
hoodieRecords.isEmpty() in HoodieSparkSqlWriter#write [19]
[Spark Integration] Unpersist the input rdd after the commit is completed
to save the memory space for inline compaction [20]
[Spark Integration] Fix Exception Cause By Table Name Case Sensitivity For
Append Mode Write [21]
[Flink Integration] Default consumes from the latest instant for flink
streaming reader [22]
[Flink Integration] Builtin sort operator for flink bulk insert [23]
[Core] Fix missing HoodieWriteStat in HoodieCreateHandle [24]



[Tests] Fixing hudi_test_suite for spark nodes and adding spark bulk_insert
node [1]
[Tests] Fix NullPointerException in TestHoodieConsoleMetrics [2]
[Tests] Refactoring few tests to reduce runningtime. DeltaStreamer and
MultiDeltaStreamer tests. Bulk insert row writer tests [3]



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message