hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <carla.stae...@nokia.com>
Subject RE: File Path and Partition names
Date Tue, 02 Oct 2012 12:56:50 GMT
Thanks Bejoy, I was kind of hoping to avoid all of the 'extra' work...it would be nice if hive
didn't include the partition name in the path creation...I was hoping that there was a 'set'
parameter/config I was missing.

Thanks
Carla

From: ext Bejoy KS [mailto:bejoy_ks@yahoo.com]
Sent: Tuesday, October 02, 2012 08:54
To: user@hive.apache.org
Subject: Re: File Path and Partition names

Hi Carla

If you like to have your custom directory structure for your partitions. You can create dirs
in hdfs of your choice , load data into them (If from another hive table then you can use
'Insert Overwrite Directory..' To populate an hdfs dir). Now you need to register this dir
as a new partition on to required table using

'Alter Table Add Parition ...'
Regards
Bejoy KS

Sent from handheld, please excuse typos.
________________________________
From: <carla.staeben@nokia.com<mailto:carla.staeben@nokia.com>>
Date: Tue, 2 Oct 2012 10:55:19 +0000
To: <user@hive.apache.org<mailto:user@hive.apache.org>>
ReplyTo: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: File Path and Partition names

Quick question about using hive to create new hdfs file paths.

Generally speaking, we like to keep our data files with a path similar to

Dataset/year/month/day/hour

I need to create a new table in hive and populate it with data from a different dataset, using
a HiveQL query.  If I do this:
CREATE EXTERNAL TABLE IF NOT EXISTS     new_table

(field1 string
,field2 string
,field3 string
)
partitioned by (reg_yr string, reg_mon string, reg_day string, reg_hour string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE  ;

And then do an insert overwrite into, I end up with this path in hdfs:

Dataset/reg_year=2012/reg_mon=10/reg_day=02/reg_hour=07

Is there an *easy* way to remove the partition name from the creation of the hdfs path?

Thanks
Carla

Mime
View raw message