hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-1108) Log newly allocated blocks
Date Fri, 28 Oct 2011 01:59:32 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Eli Collins updated HDFS-1108:

    Attachment: hdfs-1108-habranch.txt

Updated patch attached for the HDFS-1623 branch. It removes the new configuration option -
only append and shared-edits-dir based HA, using the config flag introduced in HDFS-1975 -
trigger block persistence. In shared-edits-dir based HA the log and sync are needed to ensure
the standby state is consistent in the case of fail-over.

This does not preclude adding persistent, guaranteed streams in the future, and introduces
no additional overhead for the BN-based approach. Similarly, we can unconditionally log blocks
in the future if benchmarking doesn't show significant impact.

> Log newly allocated blocks
> --------------------------
>                 Key: HDFS-1108
>                 URL: https://issues.apache.org/jira/browse/HDFS-1108
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: Todd Lipcon
>             Fix For: HA branch (HDFS-1623)
>         Attachments: HDFS-1108.patch, hdfs-1108-habranch.txt, hdfs-1108-habranch.txt,
> The current HDFS design says that newly allocated blocks for a file are not persisted
in the NN transaction log when the block is allocated. Instead, a hflush() or a close() on
the file persists the blocks into the transaction log. It would be nice if we can immediately
persist newly allocated blocks (as soon as they are allocated) for specific files.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message