hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-855) HDFS should repair corrupted files
Date Thu, 04 Jan 2007 17:13:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462257

Doug Cutting commented on HADOOP-855:

Mostly this sounds good to me.

> 1. change reportChecksumFailure parameter crc from int to FSInputStream

I'm confused by this one.  There's already an FSInputStream parameter.  In the DistributedFileSystem
implementation of this method, one can cast this to DFSInputStream and then access whatever
implementation-specific state is needed (like the datanode where the block in question resides).
 So I see no need to alter the reportChecksumFailure signature.

> HDFS should repair corrupted files
> ----------------------------------
>                 Key: HADOOP-855
>                 URL: https://issues.apache.org/jira/browse/HADOOP-855
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Wendy Chien
>         Assigned To: Wendy Chien
> While reading if we discover a mismatch between a block and checksum, we want to report
this back to the namenode to delete the corrupted block or crc.
> To implement this, we need to do the following:
> DFSInputStream
> 1. move DFSInputStream out of DFSClient
> 2. add member variable to keep track of current datanode (the chosen node)
> DistributedFileSystem
> 1. change reportChecksumFailure parameter crc from int to FSInputStream (needed to be
able to delete it). 
> 2. determine specific block and datanode from DFSInputStream passed to reportChecksumFailure
> 3. call namenode to delete block/crc vis DFSClient
> ClientProtocol
> 1. add method to ask namenode to delete certain blocks on specifc datanode.
> Namenode
> 1. add ability to delete certain blocks on specific datanode

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message