hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14333) HADOOP-14104 changed DFSClient API isHDFSEncryptionEnabled, impacted hacky hive code
Date Thu, 20 Apr 2017 20:12:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977379#comment-15977379
] 

Rushabh S Shah edited comment on HADOOP-14333 at 4/20/17 8:11 PM:
------------------------------------------------------------------

If I understood the hive code correctly, {{isEncryptionEnabled}} method is getting called
indirectly by CryptoProcessor. 
The javadoc for the {{CryptoProcessor}} says:
{noformat}
/**
 * This class processes HADOOP commands used for HDFS encryption. It is meant to be run 
 * only by Hive unit & queries tests.
 */
public class CryptoProcessor implements CommandProcessor {
{noformat}
So I assume its only test code that is breaking. In that case I wouldn't change anything from
HADOOP-14104 just to fix hive tests failure.
bq. removing the throws added by HADOOP-14104 is a quicker solution to unblock hive
Secondly removing throws and returning false in case of Exception is _wrong_ since there can
be any exception while calling getServerDefaults (like socket timeout) and we will return
false even if encryption is enabled.
It will be good to hear from members who are working on hive.
PS: I was referring to code from hive git repository master branch.



was (Author: shahrs87):
If I understood the hive code correctly, {{isEncryptionEnabled}} method is getting called
indirectly by CryptoProcessor. 
The javadoc for the {{CryptoProcessor}} says:
{noformat}
/**
 * This class processes HADOOP commands used for HDFS encryption. It is meant to be run 
 * only by Hive unit & queries tests.
 */
public class CryptoProcessor implements CommandProcessor {
{noformat}
So I assume its only test code that is breaking. In that case I wouldn't change anything from
HADOOP-14104 just to fix hive tests failure.
bq. removing the throws added by HADOOP-14104 is a quicker solution to unblock hive
Secondly removing throws and returning false in case of Exception is _wrong_ since there can
be any exception while calling getServerDefaults (like socket timeout) and we will return
false even if encryption is enabled.
It will be good to hear from members who are working on hive.


> HADOOP-14104 changed DFSClient API isHDFSEncryptionEnabled, impacted hacky hive code

> -------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14333
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14333
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>
> Though Hive should be fixed not to access DFSClient which is private to HADOOP, removing
the throws added by HADOOP-14104 is a quicker solution to unblock hive.
> Hive code
> {code}
> private boolean isEncryptionEnabled(DFSClient client, Configuration conf) {
>   try {
>     DFSClient.class.getMethod("isHDFSEncryptionEnabled");
>   } catch (NoSuchMethodException e) {
>     // the method is available since Hadoop-2.7.1
>     // if we run with an older Hadoop, check this ourselves
>     return !conf.getTrimmed(DFSConfigKeys.DFS_ENCRYPTION_KEY_PROVIDER_URI, "").isEmpty();
>   }
>   return client.isHDFSEncryptionEnabled();
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message