hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sam rash (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1102) HftpFileSystem : errors during transfer result in truncated transfer
Date Wed, 21 Apr 2010 02:45:49 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859171#action_12859171
] 

sam rash commented on HDFS-1102:
--------------------------------

actually, i was discussing this with another friend and they pointed out that we don't even
need to change how hftp works.  even w/chunked encoding, we should be able to verify on the
client since it'll send:


size1\n
<size1 bytes>

size2\n
<size2 bytes>

0

if we don't see fewer than size_N bytes or do not see the 0, we missed data.  the underlying
http client *should* handle this. if not, we can switch to:

http://hc.apache.org/

which apparently is better than using java.net.URL's underlying connection client.



> HftpFileSystem : errors during transfer result in truncated transfer
> --------------------------------------------------------------------
>
>                 Key: HDFS-1102
>                 URL: https://issues.apache.org/jira/browse/HDFS-1102
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node, hdfs client
>    Affects Versions: 0.20.1
>            Reporter: sam rash
>
> If an error occurs transferring the data over HTTP, the HftpInputStream does not know
it received fewer bytes than the file contains.  We can at least detect this and throw an
EOFException when this occurs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message