hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1034) Enhance datanode to read data and checksum file in parallel
Date Thu, 11 Mar 2010 23:26:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844270#action_12844270

Todd Lipcon commented on HDFS-1034:

Only scary thing about moving checksums to a different mountpoint is that the checksum file's
metadata will be on a different journal than the data files. This might end up fine, but it's
a little nervewracking in terms of what kind of consistency we get out of the FS - could cause
some very subtle bugs.

Have we seen these issues to be a significant bottleneck?

> Enhance datanode to read data and checksum file in parallel
> -----------------------------------------------------------
>                 Key: HDFS-1034
>                 URL: https://issues.apache.org/jira/browse/HDFS-1034
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
> In the current HDFS implementation, a read of a block issued to the datanode results
in a disk access to the checksum file followed by a disk access to the checksum file. It would
be nice to be able to do these two IOs in parallel to reduce read latency.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message