hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From P lva <ruvi...@gmail.com>
Subject How efficient is memory allocation in tez.
Date Sun, 05 Apr 2015 18:22:32 GMT
My tez query seems to error out.

I have a map join in which the smaller tables together are 200 MB and
trying to  have one block of main table be processed by one tez task.

Using the following formula to calculate the tez container size

Small table size + each block size + memory for sort + some more space for
threads + non heap

allocating 40 % of total for threads and other heap objects, 20% for non
heap.

Tried doing 1:2 and 1:3 (calculate for two blocks but set tez.am.min-size
to one block) They error out too.

What am I missing and is this even the right way of approaching the problem
?

Mime
View raw message