hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7037) Using distcp to copy data from insecure to secure cluster via hftp doesn't work (branch-2 only)
Date Tue, 15 Sep 2015 22:42:46 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746406#comment-14746406
] 

Aaron T. Myers commented on HDFS-7037:
--------------------------------------

[~wheat9] - with regard to your comment that "the security concerns remain unaddressed," could
you please respond to this point specifically:

bq. adding this capability to HFTP does not change the security semantics of Hadoop at all,
since RPC and other interfaces used for remote access already support allowing configurable
insecure fallback. This is not a security vulnerability. If it were, we should be removing
the ability to configure insecure fallback at all in Hadoop. We're not doing that, because
it was a deliberate choice to add that feature.

i.e., this change _is not changing the security level of Hadoop_, so I don't understand what
security concerns you have with this change. This change is proposing to expand the fallback
capability that already exists in other RPC interfaces to the HFTP interface.

> Using distcp to copy data from insecure to secure cluster via hftp doesn't work  (branch-2
only)
> ------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-7037
>                 URL: https://issues.apache.org/jira/browse/HDFS-7037
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security, tools
>    Affects Versions: 2.6.0
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-7037.001.patch
>
>
> This is a branch-2 only issue since hftp is only supported there. 
> Issuing "distcp hftp://<insecureCluster> hdfs://<secureCluster>" gave the
following failure exception:
> {code}
> 14/09/13 22:07:40 INFO tools.DelegationTokenFetcher: Error when dealing remote token:
> java.io.IOException: Error when dealing remote token: Internal Server Error
> 	at org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.run(DelegationTokenFetcher.java:375)
> 	at org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.getDTfromRemote(DelegationTokenFetcher.java:238)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$2.run(HftpFileSystem.java:252)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$2.run(HftpFileSystem.java:247)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.getDelegationToken(HftpFileSystem.java:247)
> 	at org.apache.hadoop.hdfs.web.TokenAspect.ensureTokenInitialized(TokenAspect.java:140)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.addDelegationTokenParam(HftpFileSystem.java:337)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.openConnection(HftpFileSystem.java:324)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:457)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:472)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.getFileStatus(HftpFileSystem.java:501)
> 	at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:57)
> 	at org.apache.hadoop.fs.Globber.glob(Globber.java:248)
> 	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1623)
> 	at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> 	at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:81)
> 	at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:342)
> 	at org.apache.hadoop.tools.DistCp.execute(DistCp.java:154)
> 	at org.apache.hadoop.tools.DistCp.run(DistCp.java:121)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.hadoop.tools.DistCp.main(DistCp.java:390)
> 14/09/13 22:07:40 WARN security.UserGroupInformation: PriviledgedActionException as:hadoopuser@xyz.COM
(auth:KERBEROS) cause:java.io.IOException: Unable to obtain remote token
> 14/09/13 22:07:40 ERROR tools.DistCp: Exception encountered 
> java.io.IOException: Unable to obtain remote token
> 	at org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.getDTfromRemote(DelegationTokenFetcher.java:249)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$2.run(HftpFileSystem.java:252)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$2.run(HftpFileSystem.java:247)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.getDelegationToken(HftpFileSystem.java:247)
> 	at org.apache.hadoop.hdfs.web.TokenAspect.ensureTokenInitialized(TokenAspect.java:140)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.addDelegationTokenParam(HftpFileSystem.java:337)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.openConnection(HftpFileSystem.java:324)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:457)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:472)
> 	at org.apache.hadoop.hdfs.web.HftpFileSystem.getFileStatus(HftpFileSystem.java:501)
> 	at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:57)
> 	at org.apache.hadoop.fs.Globber.glob(Globber.java:248)
> 	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1623)
> 	at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> 	at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:81)
> 	at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:342)
> 	at org.apache.hadoop.tools.DistCp.execute(DistCp.java:154)
> 	at org.apache.hadoop.tools.DistCp.run(DistCp.java:121)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.hadoop.tools.DistCp.main(DistCp.java:390)
> Caused by: java.io.IOException: Error when dealing remote token: Internal Server Error
> 	at org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.run(DelegationTokenFetcher.java:375)
> 	at org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.getDTfromRemote(DelegationTokenFetcher.java:238)
> 	... 22 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message