hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2764) Obtain delegation tokens for MR jobs in secure hbase setup
Date Tue, 03 Apr 2012 23:28:22 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245883#comment-13245883
] 

Phabricator commented on HIVE-2764:
-----------------------------------

enis has commented on the revision "HIVE-2764 [jira] Obtain delegation tokens for MR jobs
in secure hbase setup".

INLINE COMMENTS
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java:104
Yes, that is correct. It was never invoked.
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java:268 HadoopShims.setNullOutputFormat()
not only sets the OF, but sets the null output committer, etc. So we still need that call.
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:769 checkOS() is called
at the job submission time from the client, but FileSinkOperator.initializeOp() is called
from the task context when the job is running. So we need to initialize the output format.
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:781 I though HOF was hive-private.
You are saying that, we need this for maintaining binary BC? We are also changing the HOF
to extent OF, so all HOF implementations have to be fixed (to include the OF methods) and
recompiled.

  If for binary BC, I can use reflection to check whether the method is there.
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveNullValueSequenceFileOutputFormat.java:27 It
is a rearrange for the import statement.

REVISION DETAIL
  https://reviews.facebook.net/D2205

BRANCH
  hbase-dt-of

                
> Obtain delegation tokens for MR jobs in secure hbase setup  
> ------------------------------------------------------------
>
>                 Key: HIVE-2764
>                 URL: https://issues.apache.org/jira/browse/HIVE-2764
>             Project: Hive
>          Issue Type: Improvement
>          Components: HBase Handler, Security
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>         Attachments: HIVE-2764.D2205.1.patch, HIVE-2764.D2205.2.patch, HIVE-2764_v0.patch
>
>
> As discussed in HCATALOG-244, in a secure hbase setup with 0.92, we need to obtain delegation
tokens for hbase and save it in jobconf, so that tasks can access region servers. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message