From commits-return-10600-archive-asf-public=cust-asf.ponee.io@hudi.apache.org Mon Jan 27 02:05:47 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 66FE418065C for ; Mon, 27 Jan 2020 03:05:47 +0100 (CET) Received: (qmail 28427 invoked by uid 500); 27 Jan 2020 02:05:46 -0000 Mailing-List: contact commits-help@hudi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hudi.apache.org Delivered-To: mailing list commits@hudi.apache.org Received: (qmail 28417 invoked by uid 99); 27 Jan 2020 02:05:46 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Jan 2020 02:05:46 +0000 From: GitBox To: commits@hudi.apache.org Subject: [GitHub] [incubator-hudi] leesf commented on a change in pull request #1277: [WIP][HUDI-543] release notes for 0.5.1 Message-ID: <158009074666.16805.3348926176981632053.gitbox@gitbox.apache.org> References: In-Reply-To: Date: Mon, 27 Jan 2020 02:05:46 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit leesf commented on a change in pull request #1277: [WIP][HUDI-543] release notes for 0.5.1 URL: https://github.com/apache/incubator-hudi/pull/1277#discussion_r371047596 ########## File path: docs/_pages/releases.md ########## @@ -6,6 +6,46 @@ toc: true last_modified_at: 2019-12-30T15:59:57-04:00 --- +## [Release 0.5.1-incubating] + +### Download Information + * Source Release : [Apache Hudi(incubating) 0.5.1-incubating Source Release](https://www.apache.org/dist/incubator/hudi/0.5.1-incubating/hudi-0.5.1-incubating.src.tgz) ([asc](https://www.apache.org/dist/incubator/hudi/0.5.1-incubating/hudi-0.5.1-incubating.src.tgz.asc), [sha512](https://www.apache.org/dist/incubator/hudi/0.5.1-incubating/hudi-0.5.1-incubating.src.tgz.sha512)) + * Apache Hudi (incubating) jars corresponding to this release is available [here](https://repository.apache.org/#nexus-search;quick~hudi) + +### Release Highlights +* Dependency Version Upgrades + * Upgrade from Spark 2.1.0 to Spark 2.4.4 + * Upgrade from Avro 1.7.7 to Avro 1.8.2 + * Upgrade from Parquet 1.8.1 to Parquet 1.10.1 +* **IMPORTANT** This version requires your runtime spark version to be upgraded to 2.4+. +* Hudi now supports both Scala 2.11 and Scala 2.12, please refer to [Build with Scala 2.12](https://github.com/apache/incubator-hudi#build-with-scala-212) to build with Scala 2.12. +Also, the packages hudi-spark, hudi-utilities, hudi-spark-bundle and hudi-utilities-bundle are changed correspondingly to hudi-spark_{scala_version}, hudi-spark_{scala_version}, hudi-utilities_{scala_version}, hudi-spark-bundle_{scala_version} and hudi-utilities-bundle_{scala_version}. +Note that scala_version here is one of (2.11, 2.12). +* With 0.5.1, we added functionality to stop using renames for Hudi timeline metadata operations. This feature is automatically enabled for newly created Hudi tables. For existing tables, this feature is turned off by default. Please read this [section](deployment_link), before enabling this feature for existing hudi table. +To enable the new hudi timeline layout which avoids renames, use the write config "hoodie.timeline.layout.version=1". Alternatively, you can use "repair overwrite-hoodie-props" to append the line "hoodie.timeline.layout.version=1" to hoodie.properties. Note that in any case, upgrade hudi readers (query engines) first with 0.5.1-incubating release before upgrading writer. +* CLI supports `repair overwrite-hoodie-props` to overwrite the table's hoodie.properties with specified file. +* DeltaStreamer CLI parameter for capturing table type is changed from --storage-type to --table-type. Refer to [wiki](https://cwiki.apache.org/confluence/display/HUDI/Design+And+Architecture) with more latest terminologies. +* Configuration Value change for Kafka Reset Offset Strategies. Enum values are changed from LARGEST to LATEST, SMALLEST to EARLIEST for configuring Kafka reset offset strategies with configuration(auto.offset.reset) in deltastreamer. Review comment: > May be also call out the Kafka version change above at the top? I found we just change the artifactId from spark-streaming-kafka-0-8_2.11/2.12 to spark-streaming-kafka-0-10_2.11/2.12, the version is not changed. need to call out still ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services