hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "eric baldeschwieler (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2154) Non-interleaved checksums would optimize block transfers.
Date Thu, 31 Jan 2008 06:35:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564318#action_12564318

eric baldeschwieler commented on HADOOP-2154:

Hi Raghu,

I hear we were planning to do some rework of the read protocols in 17 to make them similar
to the new write protocols.  Correct?  I would think we would want to coordinate this work
with that.  This would imply that per client read we would ship the CRCs for all requested
bytes followed by all requested bytes for any given client request, right (or data/crc)? 

It's not clear what you are referring to by buffer in your comment.  In the final protocol,
I don't think the data node should do any CRC interleaving per client request, do you?

Is some of this discussed in another jira?  Could you provide a reference if so?

> Non-interleaved checksums would optimize block transfers.
> ---------------------------------------------------------
>                 Key: HADOOP-2154
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2154
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.14.0
>            Reporter: Konstantin Shvachko
>            Assignee: Raghu Angadi
> Currently when a block is transfered to a data-node the client interleaves data chunks
with the respective checksums. 
> This requires creating an extra copy of the original data in a new buffer interleaved
with the crcs.
> We can avoid extra copying if the data and the crc are fed to the socket one after another.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message