hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1085) hftp read failing silently
Date Tue, 06 Apr 2010 23:46:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854263#action_12854263
] 

Tsz Wo (Nicholas), SZE commented on HDFS-1085:
----------------------------------------------

CancelledKeyException is a RuntimeException.  The current codes may not handle it properly.

> hftp read  failing silently
> ---------------------------
>
>                 Key: HDFS-1085
>                 URL: https://issues.apache.org/jira/browse/HDFS-1085
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Koji Noguchi
>
> When performing a massive distcp through hftp, we saw many tasks fail with 
> {quote}
> 2010-04-06 17:56:43,005 INFO org.apache.hadoop.tools.DistCp: FAIL 2010/0/part-00032 :
java.io.IOException: File size not matched: copied 193855488 bytes (184.9m) to tmpfile (=hdfs://omehost.com:8020/somepath/part-00032)
> but expected 1710327403 bytes (1.6g) from hftp://someotherhost/somepath/part-00032
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:435)
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:543)
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:310)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>         at org.apache.hadoop.mapred.Child.main(Child.java:159)
> {quote}
> This means that read itself didn't fail but the resulted file was somehow smaller.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message