hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3212) Persist the epoch received by the JournalService
Date Fri, 06 Apr 2012 00:20:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247884#comment-13247884
] 

Suresh Srinivas commented on HDFS-3212:
---------------------------------------

bq. Currently, when FSEditLog starts a new segment, it calls journal.startLogSegment(), then
journal.logEdit(StartLogSegmentOp), then journal.logSync(). So there is a point of time when
the log segment is empty, with no transactions. If instead, we changed it so that the startLogSegment()
call was responsible for writing the first transaction (and only the first), atomically, then
we might not have a problem. We just have to make the restriction that the first transaction
of any segment is always deterministic (eg just START_LOG_SEGMENT(txid) and nothing else).

Okay, I am surprise to find this. All along, in previous discussions, I have been assuming
that JournalManager calls roll to JournalService and the startLog transaction is recorded
in JournalService. This is when epoch also gets persisted along with that record.

bq. I think it's just a matter of getting the ordering right. Before starting a log segment,
you need to fence prior writers. The fencing step is what writes down the epoch. Then, when
you create a new log segment, you tag it (eg by storing it in a directory per-epoch, or by
writing a metadata file next to it before you create the file). I think this is sufficiently
atomic.

Whether you store it in a directory per-epoch or record it in the startlogSegment record at
the beginning of the segment - they are essentially the same.
                
> Persist the epoch received by the JournalService
> ------------------------------------------------
>
>                 Key: HDFS-3212
>                 URL: https://issues.apache.org/jira/browse/HDFS-3212
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, name-node
>    Affects Versions: Shared journals (HDFS-3092)
>            Reporter: Suresh Srinivas
>
> epoch received over JournalProtocol should be persisted by JournalService.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message