hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "@Sanjiv Singh" <sanjiv.is...@gmail.com>
Subject loading in ORC from big compressed file
Date Tue, 21 Jun 2016 23:45:01 GMT
Hi ,

I have big compressed data file *my_table.dat.gz* ( approx size 100 GB)

# load staging table *STAGE_**my_table* from file *my_table.dat.gz*

HIVE>> LOAD DATA  INPATH '/var/lib/txt/*my_table.dat.gz*' OVERWRITE INTO
TABLE STAGE_my_table ;

*# insert into ORC table "my_table"*

HIVE>> INSERT INTO TABLE my_table SELECT * FROM TXT_my_table;
....
INFO  : Map 1: 0(+1)/1  Reducer 2: 0/1
....


Insertion into orc table in going on since 5-6 hours , Seems everything is
going sequential with one mapper reading complete file?

Please suggest ? help me in improving ORC table load.




Regards
Sanjiv Singh
Mob :  +091 9990-447-339

Mime
View raw message