hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1085) hftp read failing silently
Date Mon, 19 Jul 2010 20:26:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890005#action_12890005
] 

Tsz Wo (Nicholas), SZE commented on HDFS-1085:
----------------------------------------------

All the failed tests are not related to this.  The related JIRA are listed below.
- org.apache.hadoop.hdfs.TestFiHFlush.hFlushFi01_a; see HDFS-1206
- org.apache.hadoop.hdfs.TestFileAppend4; see HDFS-1306
- org.apache.hadoop.hdfs.security.token.block.TestBlockToken.testBlockTokenRpc; see HDFS-1284
- org.apache.hadoop.hdfs.server.common.TestJspHelper.testGetUgi; see HDFS-1285


> hftp read  failing silently
> ---------------------------
>
>                 Key: HDFS-1085
>                 URL: https://issues.apache.org/jira/browse/HDFS-1085
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>            Reporter: Koji Noguchi
>            Assignee: Tsz Wo (Nicholas), SZE
>             Fix For: 0.22.0
>
>         Attachments: h1085_20100713.patch, h1085_20100716b_y0.20.1xx.patch, h1085_20100716b_y0.20.1xx_with_test.patch,
h1085_20100716c.patch, h1085_20100716c_y0.20.1xx.patch, h1085_20100716d.patch, h1085_20100716d_y0.20.1xx.patch,
h1085_20100716d_y0.20.1xx_test.patch
>
>
> When performing a massive distcp through hftp, we saw many tasks fail with 
> {quote}
> 2010-04-06 17:56:43,005 INFO org.apache.hadoop.tools.DistCp: FAIL 2010/0/part-00032 :
java.io.IOException: File size not matched: copied 193855488 bytes (184.9m) to tmpfile (=hdfs://omehost.com:8020/somepath/part-00032)
> but expected 1710327403 bytes (1.6g) from hftp://someotherhost/somepath/part-00032
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:435)
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:543)
>         at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:310)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>         at org.apache.hadoop.mapred.Child.main(Child.java:159)
> {quote}
> This means that read itself didn't fail but the resulted file was somehow smaller.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message