hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sumana Sathish (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8424) distcp on raw namespace using HFTP protocol fails for TDE
Date Mon, 18 May 2015 23:07:59 GMT
Sumana Sathish created HDFS-8424:
------------------------------------

             Summary: distcp on raw namespace using HFTP protocol fails for TDE
                 Key: HDFS-8424
                 URL: https://issues.apache.org/jira/browse/HDFS-8424
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: security
    Affects Versions: 2.3.0
            Reporter: Sumana Sathish
            Priority: Critical


Distcp using HFTP protocol fails on raw namespace with the following exception 
{code}
sudo su - -c "/usr/hdp/current/hadoop-client/bin/hadoop distcp -i  -skipcrccheck -update hftp://ip-172-31-33-229.ec2.internal:50070/.reserved/raw/user/hrt_qa/srcDistcpWithTDE/smallFiles/
hdfs://ip-172-31-33-229.ec2.internal:8020/.reserved/raw/user/hrt_qa/destDistcpWithTDE/distcpedFiles/"
hdfs
2015-05-15 23:55:35,579|beaver.machine|INFO|2581|140406820472576|MainThread|15/05/15 23:55:35
INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=true, deleteMissing=false,
ignoreFailures=true, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsize',
sourceFileListing=null, sourcePaths=[hftp://ip-172-31-33-229.ec2.internal:50070/.reserved/raw/user/hrt_qa/srcDistcpWithTDE/smallFiles],
targetPath=hdfs://ip-172-31-33-229.ec2.internal:8020/.reserved/raw/user/hrt_qa/destDistcpWithTDE/distcpedFiles,
targetPathExists=false, preserveRawXattrs=false}
2015-05-15 23:55:36,829|beaver.machine|INFO|2581|140406820472576|MainThread|15/05/15 23:55:36
INFO impl.TimelineClientImpl: Timeline service address: http://ip-172-31-33-229.ec2.internal:8188/ws/v1/timeline/
2015-05-15 23:55:37,073|beaver.machine|INFO|2581|140406820472576|MainThread|15/05/15 23:55:37
INFO client.RMProxy: Connecting to ResourceManager at ip-172-31-33-229.ec2.internal/172.31.33.229:8050
2015-05-15 23:55:38,040|beaver.machine|INFO|2581|140406820472576|MainThread|15/05/15 23:55:38
ERROR tools.DistCp: Exception encountered
2015-05-15 23:55:38,040|beaver.machine|INFO|2581|140406820472576|MainThread|java.lang.UnsupportedOperationException:
HftpFileSystem doesn't support getXAttrs
2015-05-15 23:55:38,041|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.fs.FileSystem.getXAttrs(FileSystem.java:2559)
2015-05-15 23:55:38,041|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.util.DistCpUtils.toCopyListingFileStatus(DistCpUtils.java:322)
2015-05-15 23:55:38,042|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.SimpleCopyListing.doBuildListing(SimpleCopyListing.java:177)
2015-05-15 23:55:38,042|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.SimpleCopyListing.doBuildListing(SimpleCopyListing.java:140)
2015-05-15 23:55:38,043|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:84)
2015-05-15 23:55:38,043|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:90)
2015-05-15 23:55:38,044|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:84)
2015-05-15 23:55:38,044|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:382)
2015-05-15 23:55:38,045|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:181)
2015-05-15 23:55:38,045|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
2015-05-15 23:55:38,046|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
2015-05-15 23:55:38,046|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
2015-05-15 23:55:38,047|beaver.machine|INFO|2581|140406820472576|MainThread|at org.apache.hadoop.tools.DistCp.main(DistCp.java:430)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message