hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Dere <jd...@hortonworks.com>
Subject Re: Review Request 60259: HIVE-16926 LlapTaskUmbilicalExternalClient should not start new umbilical server for every fragment request
Date Fri, 30 Jun 2017 18:07:42 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60259/
-----------------------------------------------------------

(Updated June 30, 2017, 6:07 p.m.)


Review request for hive, Sergey Shelukhin and Siddharth Seth.


Changes
-------

Fix class not found errors occurring in some tests due to GenericUDTFGetSplits referencing
Tez token classes.


Bugs: HIVE-16926
    https://issues.apache.org/jira/browse/HIVE-16926


Repository: hive-git


Description
-------

Initial patch, restructured the LlapTaskUmbilicalExternalClient code a bit.
- Uses shared LLAP umbilical server rather than a new server per external client
- Retries rejected submissions (WorkSubmitter helper class)
- No more deferred cleanup (from HIVE-16652). One thing about this is that once clients are
closed/unregistered, communicator.stop() is called and it's removed from the registered list
of clients. So we might get a few warning messages about untracked taskAttemptIds coming in
during heartbeat() .. if this is undesirable we might be able to leave them in the registeredClients
list (but ignore heartbeats to them as they are tagged as closed), and remove them using the
HeartbeatCheckTask once they get too old.


Diffs (updated)
-----

  llap-client/src/java/org/apache/hadoop/hive/llap/LlapBaseRecordReader.java 7fff147 
  llap-client/src/java/org/apache/hadoop/hive/llap/SubmitWorkInfo.java 95b0ffc 
  llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java
406bdda 
  llap-client/src/java/org/apache/hadoop/hive/llap/tezplugins/helpers/LlapTaskUmbilicalServer.java
403381d 
  llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java eb93241 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java d4ec44e 


Diff: https://reviews.apache.org/r/60259/diff/3/

Changes: https://reviews.apache.org/r/60259/diff/2-3/


Testing
-------


Thanks,

Jason Dere


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message