hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bejoy...@yahoo.com
Subject Re: external partitioned table
Date Wed, 08 Feb 2012 16:16:01 GMT
Hi Koert
        As you are creating dir/sub dirs using mapreduce jobs out of hive, hive is unaware
of these sub dirs. There is no other way in such cases other than an add partition DDL to
register the dir with a hive partition. 
If you are using oozie or shell to trigger your jobs,you can accomplish  it as
-use java to come up with the correct add partition statement and write those statement(s)
into a file 
-execute the file using hive -f <fileName>

Hope it helps!..


Regards
Bejoy K S

From handheld, Please excuse typos.

-----Original Message-----
From: Koert Kuipers <koert@tresata.com>
Date: Wed, 8 Feb 2012 11:04:18 
To: <user@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: external partitioned table

hello all,

we have an external partitioned table in hive.

we add to this table by having map-reduce jobs (so not from hive) create
new subdirectories with the right format (partitionid=partitionvalue).

however hive doesn't pick them up automatically. we have to go into hive
shell and run "alter table sometable add partition
(partitionid=partitionvalue)". to make matter worse hive doesnt really lend
itself to running such an add-partition-operation from java (or for that
matter: hive doesn't lend itself to any easy programmatic manipulations...
grrr. but i will stop now before i go on a a rant).

any suggestions how to approach this? thanks!

best, koert

Mime
View raw message