hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3510) Improve FSEditLog pre-allocation
Date Mon, 10 Sep 2012 17:14:08 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452151#comment-13452151

Suresh Srinivas commented on HDFS-3510:

Colin, let me know if you are free to work on branch-1 version of this patch. If you are busy,
I can see if someone else can pick it up.
> Improve FSEditLog pre-allocation
> --------------------------------
>                 Key: HDFS-3510
>                 URL: https://issues.apache.org/jira/browse/HDFS-3510
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>             Fix For: 3.0.0
>         Attachments: HDFS-3510.001.patch, HDFS-3510.003.patch, HDFS-3510.004.patch, HDFS-3510.004.patch,
HDFS-3510.006.patch, HDFS-3510.007.patch, HDFS-3510.008.patch, HDFS-3510.009.patch, HDFS-3510.010.patch,
HDFS-3510.011.patch, HDFS-3510-b1.001.patch, HDFS-3510-b1.002.patch
> It is good to avoid running out of space in the middle of writing a batch of edits, because
when it happens, we often get partial edits at the end of the log.
> Edit log preallocation can solve this problem (see HADOOP-2330 for a full description
of edit log preallocation).
> The current pre-allocation code was introduced for performance reasons, not for preventing
partial edits.  As a consequence, we sometimes do a write without using pre-allocation.  We
should change the pre-allocation code so that it always preallocates at least enough space
before writing out the edits.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message