hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Satish Mittal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6109) Support customized location for EXTERNAL tables created by Dynamic Partitioning
Date Thu, 26 Dec 2013 13:08:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13856862#comment-13856862
] 

Satish Mittal commented on HIVE-6109:
-------------------------------------

As an example, suppose a table user_logs is partitioned by (year, month, day, hour, minute,
country) and stored at location "hdfs://hcat/data/user_logs". 

Currently dynamic partitions for user_logs would get created at HDFS locations in the fixed
format "hdfs://hcat/data/user_logs/year=2013/month=12/hour=06/minute=10/country=US". However
in our use-case this breaks other data pipeline applications which expect the data to arrive
in a different format: "hdfs://hcat/data/user_logs/2013/12/06/10/US". There could be similar
other use-cases possible.

> Support customized location for EXTERNAL tables created by Dynamic Partitioning
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-6109
>                 URL: https://issues.apache.org/jira/browse/HIVE-6109
>             Project: Hive
>          Issue Type: Improvement
>          Components: HCatalog
>            Reporter: Satish Mittal
>
> Currently when dynamic partitions are created by HCatalog, the underlying directories
for the partitions are created in a fixed 'Hive-style' format, i.e. root_dir/key1=value1/key2=value2/....
and so on. However in case of external table, user should be able to control the format of
directories created for dynamic partitions.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message