hudi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Udit Mehrotra <udi...@apache.org>
Subject [DISCUSS] Hudi 0.9.0 Release
Date Wed, 04 Aug 2021 00:12:45 GMT
Hi Community,

As we draw close to doing Hudi 0.9.0 release, I am happy to share a summary
of the key features/improvements that would be going in the release and the
current blockers for everyone's visibility.

*Highlights*

   - [HUDI-1729] Asynchronous Hive sync and commits cleaning for Flink
   writer
   - [HUDI-1738] Detect and emit deleted records for Flink MOR table
   streaming read
   - [HUDI-1867] Support streaming reads for Flink COW table
   - [HUDI-1908] Global index for flink writer
   - [HUDI-1788] Support Insert Overwrite with Flink Writer
   - [HUDI-2209] Bulk insert for flink writer
   - [HUDI-1591] Support querying using non-globbed paths for Hudi Spark
   DataSource queries
   - [HUDI-1591] Partition pruning support for read optimized queries via
   Hudi Spark DataSource
   - [HUDI-1415] Register Hudi Table as a Spark DataSource Table with
   metastore. Queries via Spark SQL will be routed through Hudi DataSource
   (instead of InputFormat), thus making it more performant due to Spark's
   native/optimized readers
   - [HUDI-1591] Partition pruning support for snapshot queries via Hudi
   Spark DataSource
   - [HUDI-1658] DML and DDL support via Spark SQL
   - [HUDI-1790] Add SqlSource for DeltaStreamer to support backfill use
   cases:
   - [HUDI-251] Add JDBC Source support for DeltaStreamer
   - [HUDI-1910] Support Kafka based checkpointing for HoodieDeltaStreamer
   - [HUDI-1371] Support metadata based listing for Spark DataSource and
   Spark SQL
   - [HUDI-2013] [HUDI-1717] [HUDI-2089] [HUDI-2016] Improvements to
   Metadata based listing
   - HUDI-89] Introduce a HoodieConfig/ConfigProperty framework to bring
   all configs under one roof
   - [HUDI-2124] Grafana dashboard for Hudi
   - [HUDI-1104] [HUDI-1105] [HUDI-2009] Improvements to Bulk Insert via
   row writing
   - [HUDI-1483] Async clustering for Delta Streamer
   - [HUDI-2235] Add virtual key support to Hudi
   - [HUDI-1848] Add support for Hive Metastore in Hive-sync-tool
   - In addition, there have been significant improvements and bug fixes to
   improve the overall stability of Flink Hudi integration

*Current Blockers*

   - [HUDI-2208] Support Bulk Insert For Spark Sql (Owner: pengzhiwei)
   - [HUDI-1256] Follow on improvements to HFile tables for metadata based
   listing (Owner: None)
   - [HUDI-2063] Add Doc For Spark Sql (DML and DDL) integration With Hudi
   (Owner: pengzhiwei)
   - [HUDI-1842] Spark Sql Support For The Exists Hoodie Table (Owner:
   pengzhiwei)
   - [HUDI-1138] Re-implement marker files via timeline server (Owner:
   Ethan Guo)
   - [HUDI-1985] Website redesign implementation (Owner: Vinoth
   Govindarajan)
   - [HUDI-2232] MERGE INTO fails with table having nested struct (Owner:
   pengzhiwei)
   - [HUDI-1468] incremental read support with clustering (Owner: Liwei)
   - [HUDI-2250] Bulk insert support for tables w/ primary key (Owner: None)
   - [HUDI-2222] [SQL] Test catalog integration (Owner: Sagar Sumit)
   - [HUDI-2221] [SQL] Functionality testing with Spark 2 (Owner: Sagar
   Sumit)
   - [HUDI-1887] Setting default value to false for enabling schema post
   processor (Owner: Sivabalan)
   - [HUDI-1850] Fixing read of a empty table but with failed write (Owner:
   Sivabalan)
   - [HUDI-2151] Enable defaults for out of box performance (Owner: Udit
   Mehrotra)
   - [HUDI-2119] Ensure the rolled-back instance was previously synced to
   the Metadata Table when syncing a Rollback Instant (Owner: Prashant Wason)
   - [HUDI-1458] Support custom clustering strategies and preserve commit
   time to support incremental read (Owner: Satish Kotha)
   - [HUDI-1763] Fixing honoring of Ordering val in
   DefaultHoodieRecordPayload.preCombine (Owner: Sivabalan)
   - [HUDI-1129] Improving schema evolution support in hudi (Owner:
   Sivabalan)
   - [HUDI-2120] [DOC] Update docs about schema in flink sql configuration
   (Owner: Xianghu Wang)
   - [HUDI-2182] Support Compaction Command For Spark Sql (Owner:
   pengzhiwei)

Please respond to the thread if you think that I have missed capturing any
of the highlights or blockers for Hudi 0.9.0 release. For the owners of
these release blockers, can you please provide a specific timeline you are
willing to commit to for finishing these so we can cut an RC ?

Thanks,
Udit

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message