hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahender Sarangam <>
Subject Hive ORC Table
Date Sat, 21 Jan 2017 00:58:18 GMT
Hi All,

We have ORC table which is of 2 GB size. When we perform operation on 
top of this ORC table, Tez always deduce 1009 reducer every time. I 
searched 1009 is considered as Maximum value of number of Tez task. Is 
there a way to reduce the number of reducer. I see file generated 
underlying ORC some of them 500 MB or 1 GB etc. Is there way to 
distribute file size to same value/same size.

My Second scenario, we have join on 5 tables all of them are left join. 
Query goes fast till reached 99%. From 99% to 100% it takes too much 
time. We are not involving our partition column as part of LEFT JOIN 
Statement, Is there better way to resolving issues on 99% hanging 
condition. My table is of 20 GB we are left joining with another table ( 
9,00,00,000) records.


View raw message