Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3A3A0D5A8 for ; Thu, 13 Sep 2012 03:05:29 +0000 (UTC) Received: (qmail 88527 invoked by uid 500); 13 Sep 2012 03:05:24 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 88166 invoked by uid 500); 13 Sep 2012 03:05:24 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 88152 invoked by uid 99); 13 Sep 2012 03:05:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Sep 2012 03:05:23 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [198.160.100.225] (HELO mx1.acxiom.com) (198.160.100.225) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 13 Sep 2012 03:05:15 +0000 x-bypass-officex: False X-IronPort-AV: E=Sophos;i="4.80,413,1344229200"; d="scan'208,217";a="10436534" Received: from unknown (HELO LITIGHCCRP02.Corp.Acxiom.net) ([10.98.27.31]) by mx1out.acxiom.com with ESMTP; 12 Sep 2012 22:02:16 -0500 Received: from CWYIGMBCRP01.Corp.Acxiom.net ([fe80::7828:a9c4:1647:a1ea]) by LITIGHCCRP02.Corp.Acxiom.net ([::1]) with mapi id 14.01.0289.001; Wed, 12 Sep 2012 22:04:53 -0500 From: Nataraj Rashmi - rnatar To: "user@hadoop.apache.org" , "bejoy.hadoop@gmail.com" Subject: RE: How to make the hive external table read from subdirectories Thread-Topic: How to make the hive external table read from subdirectories Thread-Index: Ac2RGWIPKxztul6oTXOIWQGo3WTD/QAMu/EAAANhiFA= Date: Thu, 13 Sep 2012 03:04:52 +0000 Message-ID: References: <287915395-1347480520-cardhu_decombobulator_blackberry.rim.net-1587235649-@b25.c15.bise7.blackberry> In-Reply-To: <287915395-1347480520-cardhu_decombobulator_blackberry.rim.net-1587235649-@b25.c15.bise7.blackberry> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [139.61.238.11] Content-Type: multipart/alternative; boundary="_000_A67F253AF2070D4EB9822234EE6EE06EB44C7CAECWYIGMBCRP01Cor_" MIME-Version: 1.0 --_000_A67F253AF2070D4EB9822234EE6EE06EB44C7CAECWYIGMBCRP01Cor_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Thanks for your response. Can someone see if this is ok? I am not getting a= ny records when I query the hive table when I use Partitions. This is how I am creating the table. CREATE EXTERNAL TABLE Data (field1 STRING,field2....) PARTITIONED BY(year S= TRING, month STRING, dayofmonth STRING) ROW FORMAT DELIMITED FIELDS TERMINA= TED BY '|' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/myus= er/MapReduceOutput'; Data dir looks like this. /user/myuser/MapReduceOutput/2012/09/11 When I create the table using '/user/myuser/MapReduceOutput/2012/09/11' as = the location, I can query the table and get data back. Please advice, Thanks. From: Bejoy KS [mailto:bejoy.hadoop@gmail.com] Sent: Wednesday, September 12, 2012 3:09 PM To: user@hadoop.apache.org Subject: Re: How to make the hive external table read from subdirectories Hi Natraj Create a partitioned table and add the sub dirs as partitions. You need to = have some logic in place for determining the partitions. Say if the sub dir= s denote data based on a date then make date as the partition. Regards Bejoy KS Sent from handheld, please excuse typos. ________________________________ From: Nataraj Rashmi - rnatar Date: Wed, 12 Sep 2012 19:19:19 +0000 To: user@hadoop.apache.org ReplyTo: user@hadoop.apache.org Subject: How to make the hive external table read from subdirectories I have a hive external table created from a hdfs location. How do I make it= read the data from all the subdirectories also? Thanks. *************************************************************************** The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank You. ***************************************************************************= * --_000_A67F253AF2070D4EB9822234EE6EE06EB44C7CAECWYIGMBCRP01Cor_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Thanks for your respon= se. Can someone see if this is ok? I am not getting any records when I quer= y the hive table when I use Partitions.

 

This is how I am creat= ing the table.

CREATE EXTERNAL TABLE = Data (field1 STRING,field2….) PARTITIONED BY(year STRING, month STRIN= G, dayofmonth STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES T= ERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/myuser/MapReduceOutput';

 

Data dir looks  l= ike this.

/user/myuser/MapReduce= Output/2012/09/11

 

When I create the tabl= e using ‘/user/myuser/MapReduceOutput/2012/09/11’ as the locati= on, I can query the table and get data back.

 

Please advice,

Thanks.

 

 

From: Bejoy KS= [mailto:bejoy.hadoop@gmail.com]
Sent: Wednesday, September 12, 2012 3:09 PM
To: user@hadoop.apache.org
Subject: Re: How to make the hive external table read from subdirect= ories

 

Hi Na= traj

Create a partitioned table and add the sub dirs as partitions. You need to = have some logic in place for determining the partitions. Say if the sub dir= s denote data based on a date then make date as the partition.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

From: Nataraj Rashmi - rnatar <Rashmi.Nataraj@acxio= m.com>

Date: Wed, 12 Sep 2012 19:19:19 +0000

To: user@hadoop.apache.org<user@hadoop.apache.org= >

ReplyTo: user@hadoop.apache.org

Subject: How to make the hive external table read from su= bdirectories

 

I have a hive external table created from a hdfs loc= ation. How do I make it read the data from all  the subdirectories als= o?

 

Thanks.

************************************************************************= ***
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally<= br> privileged.

If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.

If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.

Thank You.
***************************************************************************= *

--_000_A67F253AF2070D4EB9822234EE6EE06EB44C7CAECWYIGMBCRP01Cor_--