hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nataraj Rashmi - rnatar <Rashmi.Nata...@acxiom.com>
Subject RE: How to make the hive external table read from subdirectories
Date Thu, 13 Sep 2012 03:04:52 GMT
Thanks for your response. Can someone see if this is ok? I am not getting any records when
I query the hive table when I use Partitions.

This is how I am creating the table.
CREATE EXTERNAL TABLE Data (field1 STRING,field2....) PARTITIONED BY(year STRING, month STRING,
dayofmonth STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'
STORED AS TEXTFILE LOCATION '/user/myuser/MapReduceOutput';

Data dir looks  like this.
/user/myuser/MapReduceOutput/2012/09/11

When I create the table using '/user/myuser/MapReduceOutput/2012/09/11' as the location, I
can query the table and get data back.

Please advice,
Thanks.


From: Bejoy KS [mailto:bejoy.hadoop@gmail.com]
Sent: Wednesday, September 12, 2012 3:09 PM
To: user@hadoop.apache.org
Subject: Re: How to make the hive external table read from subdirectories

Hi Natraj

Create a partitioned table and add the sub dirs as partitions. You need to have some logic
in place for determining the partitions. Say if the sub dirs denote data based on a date then
make date as the partition.
Regards
Bejoy KS

Sent from handheld, please excuse typos.
________________________________
From: Nataraj Rashmi - rnatar <Rashmi.Nataraj@acxiom.com>
Date: Wed, 12 Sep 2012 19:19:19 +0000
To: user@hadoop.apache.org<user@hadoop.apache.org>
ReplyTo: user@hadoop.apache.org
Subject: How to make the hive external table read from subdirectories

I have a hive external table created from a hdfs location. How do I make it read the data
from all  the subdirectories also?

Thanks.

***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.

If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.

If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.

Thank You.
****************************************************************************

Mime
View raw message