hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <>
Subject [jira] [Commented] (HIVE-11355) Hive on tez: memory manager for sort buffers (input/output) and operators
Date Fri, 07 Aug 2015 19:49:46 GMT


Sergey Shelukhin commented on HIVE-11355:

Some comments in evaluateUnionWork? Input size is computed from a subset of output edges,
but no input edges are taken into account
Comment for "forAll". Actually I don't understand the difference between using regex rule
for mapjoins and the default rule that checks for mapjoins. Aren't they supposed to do the
same thing?
           throw new SemanticException("Memory shortage of " + -(totalAvailableMemory)
                + ". Please modify the container size to be greater than "
isn't it supposed to already be addressed by the above exception? At least, the message seems
to indicate so.
pctx.conf.getLongVar(HiveConf.ConfVars.MAPREDMAXSPLITSIZE - can this be 0?
if (edge.getDataFlowSize() > TEN_MB) { nit - indentation.
To be continued
Actually RB would help :)

> Hive on tez: memory manager for sort buffers (input/output) and operators
> -------------------------------------------------------------------------
>                 Key: HIVE-11355
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Tez
>    Affects Versions: 2.0.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>         Attachments: HIVE-11355.1.patch, HIVE-11355.2.patch, HIVE-11355.3.patch
> We need to better manage the sort buffer allocations to ensure better performance. Also,
we need to provide configurations to certain operators to stay within memory limits.

This message was sent by Atlassian JIRA

View raw message