hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erix Yao <>
Subject Re: can I use hive dynamic partition while loading data into tables?
Date Fri, 15 Apr 2011 07:31:20 GMT
Does this mean if I want the type field as the partition key , I will have
to split the raw data by myself and load the files into the target table?

I see there's an example in tutorial:

FROM page_view_stg pvs
    INSERT OVERWRITE TABLE page_view PARTITION(dt='2008-06-08', country)
           SELECT pvs.viewTime, pvs.userid, pvs.page_url,
pvs.referrer_url, null, null, pvs.ip,

but insert must overwrite the table whole table partition.
Can I insert without the overwrite key word?

2011/4/15 Ning Zhang <>

>  The LOAD DATA command only copy the files to the destination directory. It
> doesn't read the records of the input file, so it cannot do partitioning
> based on record values.
>  On Apr 14, 2011, at 10:52 PM, Erix Yao wrote:
> hi,all
>     The dynamic partition function is amazing ,but only works in insert
> clause. Can I use it while loading data into table?
>      For example: load data  LOAD DATA LOCAL INPATH
> `/tmp/pv_2008-06-08_us.txt` INTO TABLE page_view
> PARTITION(date='2008-06-08', country='US', type);
> type is the dynamic partition key in the raw data?
>  This will be very cool! If is supported, I will not have to category the
> raw data according to the type column.
> --
> haitao.yao@Beijing


View raw message