hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jasper Knulst <jasper.knu...@incentro.com>
Subject Re: Partitioning EXTERNAL TABLE without copying or moving files
Date Thu, 08 Dec 2011 23:59:44 GMT
Hi Vince,

Hive partitioning can only exist by issueing new directories in HDFS. There
is no way to partition the data in a Hive table without adding extra
filepaths/dirs in HDFS.

For an external table you have to redistribute the data yourself in
corresponding filepaths and add the new partition based on that by editing
the Hive metadata.

Cheers Jasper
Op 8 dec. 2011 21:47 schreef "Vince Hoang" <vhoang@cafepress.com> het
volgende:

>   Hi,
>
>  I am running Hive 0.7.0 with Hadoop 0.20.2.  I have one HDFS folder full
> of web server logs dated back several months.
>
>  Is possible to partition an EXTERNAL TABLE without copying/moving files
> or altering the layout of the directory?
>
>  For example, in HDFS, I have:
>
>  > /logs/log-2011-09-01
>  > /logs/log-2011-09-02
>  >   …
>  > /logs/log-2011-12-01
>
>  I'd like to know if it's possible to partition the EXTERNAL TABLE by
> date without having to create subdirectories:
>
>  > /logs/2011-09-01/log-2011-09-01
>  > /logs/2011-09-02/log-2011-09-02
>  >   …
>  > /logs/2011-12-01/log-2011-12-01
>
>  Is it possible?
>
>   Thanks,
> Vince
>
>
> The contents of this message, together with any attachments, are intended
> only for the use of the individual or entity to which they are addressed
> and may contain information that is confidential and exempt from
> disclosure. If you are not the intended recipient, you are hereby notified
> that any dissemination, distribution, or copying of this message, or any
> attachment, is strictly prohibited. If you have received this message in
> error, please notify the original sender immediately by telephone or by
> return E-mail and delete this message, along with any attachments, from
> your computer. Thank you.
>

Mime
View raw message