hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2392) Dist with hftp is failing again
Date Wed, 05 Oct 2011 13:36:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120963#comment-13120963
] 

Daryn Sharp commented on HDFS-2392:
-----------------------------------

I'll include it in the trunk merge of HADOOP-7510 since it was an oversight in that jira's
changes.
                
> Dist with hftp is failing again
> -------------------------------
>
>                 Key: HDFS-2392
>                 URL: https://issues.apache.org/jira/browse/HDFS-2392
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.20.205.0
>            Reporter: Rajit Saha
>            Assignee: Daryn Sharp
>            Priority: Critical
>             Fix For: 0.20.205.0
>
>         Attachments: HDFS-2392.patch
>
>
> $ hadoop distcp hftp://<NN Hostname>:50070/user/hadoopqa/input1/part-00000 /user/hadoopqa/out3
> 11/09/30 18:57:59 INFO tools.DistCp: srcPaths=[hftp://<NN Hostname>:50070/user/hadoopqa/input1/part-00000]
> 11/09/30 18:57:59 INFO tools.DistCp: destPath=/user/hadoopqa/out3
> 11/09/30 18:58:00 INFO security.TokenCache: Got dt for
> hftp://<NN Hostname>:50070/user/hadoopqa/input1/part-00000;uri=<NN IP>:50470;t.service=<NN
IP>:50470
> 11/09/30 18:58:00 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 24 for hadoopqa
on <NN IP>:8020
> 11/09/30 18:58:00 INFO security.TokenCache: Got dt for
> /user/hadoopqa/out3;uri=<NN IP>:8020;t.service=<NN IP>:8020
> 11/09/30 18:58:00 INFO tools.DistCp: /user/hadoopqa/out3 does not exist.
> 11/09/30 18:58:00 INFO tools.DistCp: sourcePathsCount=1
> 11/09/30 18:58:00 INFO tools.DistCp: filesToCopyCount=1
> 11/09/30 18:58:00 INFO tools.DistCp: bytesToCopyCount=1.0g
> 11/09/30 18:58:01 INFO mapred.JobClient: Running job: job_201109300819_0007
> 11/09/30 18:58:02 INFO mapred.JobClient:  map 0% reduce 0%
> 11/09/30 18:58:25 INFO mapred.JobClient: Task Id : attempt_201109300819_0007_m_000000_0,
Status : FAILED
> java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 11/09/30 18:58:41 INFO mapred.JobClient: Task Id : attempt_201109300819_0007_m_000000_1,
Status : FAILED
> java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 11/09/30 18:58:56 INFO mapred.JobClient: Task Id : attempt_201109300819_0007_m_000000_2,
Status : FAILED
> java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 11/09/30 18:59:14 INFO mapred.JobClient: Job complete: job_201109300819_0007
> 11/09/30 18:59:14 INFO mapred.JobClient: Counters: 6
> 11/09/30 18:59:14 INFO mapred.JobClient:   Job Counters 
> 11/09/30 18:59:14 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=62380
> 11/09/30 18:59:14 INFO mapred.JobClient:     Total time spent by all reduces waiting
after reserving slots (ms)=0
> 11/09/30 18:59:14 INFO mapred.JobClient:     Total time spent by all maps waiting after
reserving slots (ms)=0
> 11/09/30 18:59:14 INFO mapred.JobClient:     Launched map tasks=4
> 11/09/30 18:59:14 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
> 11/09/30 18:59:14 INFO mapred.JobClient:     Failed map tasks=1
> 11/09/30 18:59:14 INFO mapred.JobClient: Job Failed: # of failed Map Tasks exceeded allowed
limit. FailedCount: 1.
> LastFailedTask: task_201109300819_0007_m_000000
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.io.IOException: Job failed!
>         at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1257)
>         at org.apache.hadoop.tools.DistCp.copy(DistCp.java:667)
>         at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>         at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message