hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasad Mujumdar" <pras...@cloudera.com>
Subject Review Request 14056: HIVE-5246: Local task for map join submitted via oozie job fails on a secure HDFS
Date Tue, 10 Sep 2013 02:25:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14056/
-----------------------------------------------------------

Review request for hive, Gunther Hagleitner and Thejas Nair.


Bugs: HIVE-5246
    https://issues.apache.org/jira/browse/HIVE-5246


Repository: hive-git


Description
-------

For a Hive query started by Oozie Hive action, the local task submitted for Mapjoin fails.
The HDFS delegation token is not shared properly with the child JVM created for the local
task.
Oozie creates a delegation token for the Hive action and sets env variable HADOOP_TOKEN_FILE_LOCATION
as well as mapreduce.job.credentials.binary config property. However this doesn't get passed
down to the child JVM which causes the problem.

This patch is modifying HIVE-4343 to load the token file path from env variable which will
be set for both HiveServer2 (by original HIVE-4343 patch) as well as Oozie.


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/SecureCmdDoAs.java 8124639 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 4cc7129 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 341154d 

Diff: https://reviews.apache.org/r/14056/diff/


Testing
-------

A test application attached to the ticket (https://issues.apache.org/jira/secure/attachment/12602266/HIVE-5246-test.tar)
that simulates oozie behavior.
Manually tested HS2 to verify that it doesn't regress HIVE-4343


Thanks,

Prasad Mujumdar


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message