hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bejoy...@yahoo.com
Subject Re: how to load data to partitioned table
Date Fri, 12 Aug 2011 13:58:05 GMT
Hi Daniel
      Just having a look at your requirement , to load data into a partition based hive table
from any input file the most hassle free approach would be.
1.  Load the data into a non partitioned table that shares similar structure as the target
table.
2. Populate the target table with the data from non partitioned one using hive dynamic partition
approach.
With Dynamic partitions you don't need to manually identify the data partitions and distribute
data accordingly. 

A similar implementation is described in the blog post
www.kickstarthadoop.blogspot.com/2011/06/how-to-speed-up-your-hive-queries-in.html

Hope it helps

Regards
Bejoy K S

-----Original Message-----
From: Vikas Srivastava <vikas.srivastava@one97.net>
Date: Fri, 12 Aug 2011 17:31:28 
To: <user@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: Re: how to load data to partitioned table

Hey ,

Simpley you have run query like this

FROM sales_temp INSERT OVERWRITE TABLE sales partition(period_key) SELECT *


Regards
Vikas Srivastava


2011/8/12 Daniel,Wu <hadoop_wu@163.com>

>   suppose the table is partitioned by period_key, and the csv file also has
> a column named as period_key. The csv file contains multiple days of data,
> how can we load it in the the table?
>
> I think of an workaround by first load the data into a non-partition table,
> and then insert the data from non-partition table to the partition table.
>
> hive> INSERT OVERWRITE TABLE sales SELECT * FROM sales_temp;
> FAILED: Error in semantic analysis: need to specify partition columns
> because the destination table is partitioned.
>
>
> However it doesn't work also. please help.
>
>
>


-- 
With Regards
Vikas Srivastava

DWH & Analytics Team
Mob:+91 9560885900
One97 | Let's get talking !

Mime
View raw message