hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagat <>
Subject How you manage inserts into dynamic partitions
Date Fri, 20 Apr 2012 15:08:26 GMT
Hello Everyone ,

We have a pig based ETL process and output of those are processed csv files.

Now just want to learn about best practices on how you guys insert the data
in dynamically generated partitions , do you use some shell script ? or
some other utility to do the insertion ?

How you handle situations like a csv having data which should go to two
separate partitions in hive ? Do you use Pig spilt to do this kind of
division or outside pig.

Just curious to know best practices to handle insertion into dynamic
partitions , when data has been partitioned on time factor ( e,g Month ,
Week , Day etc)

Have a great weekend ahead,

Thanks and Regards,

Jagat Singh

View raw message