Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 15 Oct 2013 17:40:46 +0000 (UTC)
From: "Brock Noland (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12662210.1375819557545.67185.1381858846505@arcas>
In-Reply-To: <JIRA.12662210.1375819557545@arcas>
References: <JIRA.12662210.1375819557545@arcas>
Subject: [jira] [Updated] (HIVE-5011) Dynamic partitioning in HCatalog
 broken on external tables
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HIVE-5011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brock Noland updated HIVE-5011:
-------------------------------

    Fix Version/s: 0.12.0

This was fixed in 0.12.

> Dynamic partitioning in HCatalog broken on external tables
> ----------------------------------------------------------
>
>                 Key: HIVE-5011
>                 URL: https://issues.apache.org/jira/browse/HIVE-5011
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>            Priority: Critical
>             Fix For: 0.12.0
>
>         Attachments: HIVE-5011.patch
>
>
> Dynamic partitioning with HCatalog has been broken as a result of HCATALOG-500 trying to support user-set paths for external tables.
> The goal there was to be able to support other custom destinations apart from the normal "hive-style" partitions. However, it is not currently possible for users to set paths for dynamic ptn writes, since we don't support any way for users to specify "patterns"(like, say "$\{rootdir\}/$v1.$v2/") into which writes happen, only "locations", and the values for dyn. partitions are not known ahead of time. Also, specifying a custom path messes with the way dynamic ptn. code tries to determine what was written to where from the output committer, which means that even if we supported patterned-writes instead of location-writes, we still have to do some more deep diving into the output committer code to support it.
> Thus, my current proposal is that we honour writes to user-specified paths for external tables *ONLY* for static partition writes - i.e., if we can determine that the write is a dyn. ptn. write, we will ignore the user specification. (Note that this does not mean we ignore the table's external location - we honour that - we just don't honour any HCatStorer/etc provided additional location - we stick to what metadata tells us the root location is.


--
This message was sent by Atlassian JIRA
(v6.1#6144)