hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
Date Wed, 01 Jun 2011 17:36:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042313#comment-13042313

Ivan Kelly commented on HDFS-1580:

(1) should work for checkpointing, as if you journal A has more edits than journal B while
counting the in_progress file, it will have more or an equal number not counting the in_progress
file. More in the case that B has gaps in which case it throws an exception, equal otherwise.

So we finalise inprogress when we open a write and spot an inprogress file. I guess this should
only happen on startup after a crash. The writer shouldn't finalise an inprogress if something
else is writing to it. We have nothing to prevent this now, but if this is happening, your
system is broken. Fencing could be implemented later to explicitly exclude this possibility.

> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>                 Key: HDFS-1580
>                 URL: https://issues.apache.org/jira/browse/HDFS-1580
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>             Fix For: Edit log branch (HDFS-1073)
>         Attachments: EditlogInterface.1.pdf, EditlogInterface.2.pdf, HDFS-1580+1521.diff,
HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf,
generic_wal_iface.pdf, generic_wal_iface.txt

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message