hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1908) Restructure data node code so that block sending/receiving is seperated from data transfer header handling
Date Wed, 19 Sep 2007 17:35:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12528836
] 

dhruba borthakur commented on HADOOP-1908:
------------------------------------------

I think that a datanode should report an error if the next datanode in the pipeline reported
an error. This allows the client to detect if it failed to create the desired number of replicas.


The recovery from this error is simple. The client abandons the entire block, fetches a new
blockid and block locations from the namenode and restarts flushing the current block.

> Restructure data node code so that block sending/receiving is seperated from data transfer
header handling
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1908
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1908
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.14.1
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.15.0
>
>         Attachments: datanode.patch
>
>
> This jira is intended for code sharing. I'd like to have a BlockSender which is resposible
for reading a block from a disk and writing it to an output stream, and a BlockReceiver which
receives a block from an input stream, writes it to a disk,  and possibly writes it a mirror
ouput stream to support pipeline writes. Block sender and receiver code is independent of
any block IO protocols so it could be shared by block transfer, block write, block read, and
future block replacement introduced by data node rebalancing.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message