hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Navis Ryu" <navis....@nexr.com>
Subject Re: Review Request 37347: HIVE-11518 Provide interface to adjust required resource for tez tasks
Date Tue, 13 Oct 2015 01:48:09 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37347/
-----------------------------------------------------------

(Updated 10 13, 2015, 1:48 오전)


Review request for hive.


Changes
-------

Rebased to trunk


Bugs: HIVE-11518
    https://issues.apache.org/jira/browse/HIVE-11518


Repository: hive-git


Description
-------

Resource requirements for each tasks are varied but currently it's fixed to one value(via
hive.tez.container.size). It would be good to customize resource requirements appropriate
to expected work.

Suggested interface is quite simple.
{code}
public interface ResourceCalculator {

  Resource adjust(Resource resource, MapWork mapWork);

  Resource adjust(Resource resource, ReduceWork reduceWork);
}
{code}


Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0fcd39b 
  common/src/java/org/apache/hive/common/util/ReflectionUtil.java 7059309 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java d250b82 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ResourceCalculator.java PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/SimpleCalculator.java PRE-CREATION 

Diff: https://reviews.apache.org/r/37347/diff/


Testing
-------


Thanks,

Navis Ryu


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message