From commits-return-10945-archive-asf-public=cust-asf.ponee.io@hudi.apache.org Sun Feb 2 23:42:05 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 56F9718067E for ; Mon, 3 Feb 2020 00:42:05 +0100 (CET) Received: (qmail 12305 invoked by uid 500); 2 Feb 2020 23:42:04 -0000 Mailing-List: contact commits-help@hudi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hudi.apache.org Delivered-To: mailing list commits@hudi.apache.org Received: (qmail 12282 invoked by uid 99); 2 Feb 2020 23:42:04 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 02 Feb 2020 23:42:04 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 13CD4E2DEB for ; Sun, 2 Feb 2020 23:42:04 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id A966E78034B for ; Sun, 2 Feb 2020 23:42:02 +0000 (UTC) Date: Sun, 2 Feb 2020 23:42:02 +0000 (UTC) From: "Suneel Marthi (Jira)" To: commits@hudi.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HUDI-519) Document the need for Avro dependency shading/relocation for custom payloads, need for spark-avro MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HUDI-519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-519: ------------------------------- Fix Version/s: (was: 0.5.1) 0.5.2 > Document the need for Avro dependency shading/relocation for custom payloads, need for spark-avro > ------------------------------------------------------------------------------------------------- > > Key: HUDI-519 > URL: https://issues.apache.org/jira/browse/HUDI-519 > Project: Apache Hudi (incubating) > Issue Type: Task > Components: Docs, Usability > Reporter: Udit Mehrotra > Priority: Major > Fix For: 0.5.2 > > > In [https://github.com/apache/incubator-hudi/pull/1005] we are migrating Hudi to Spark 2.4.4. As part of this migration, we also had to migrate Hudi to use Avro 1.8.2 (required by spark), while Hive still uses older version of Avro. > This has resulted in the need to shade Avro in *hadoop-mr-bundle*. This has implications on users of Hudi, who implement custom record payloads. They would have start shading Avro in there custom jars, similar to how it shaded in *hadoop-mr-bundle*. > This Jira is to track the documentation of this caveat in release notes, and if needed at other places like website etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)