Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 6 Apr 2012 00:20:23 +0000 (UTC)
From: "Suresh Srinivas (Commented) (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <467803258.20659.1333671623212.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <396853591.20411.1333668143613.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-3212) Persist the epoch received by the
 JournalService
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247884#comment-13247884 ] 

Suresh Srinivas commented on HDFS-3212:
---------------------------------------

bq. Currently, when FSEditLog starts a new segment, it calls journal.startLogSegment(), then journal.logEdit(StartLogSegmentOp), then journal.logSync(). So there is a point of time when the log segment is empty, with no transactions. If instead, we changed it so that the startLogSegment() call was responsible for writing the first transaction (and only the first), atomically, then we might not have a problem. We just have to make the restriction that the first transaction of any segment is always deterministic (eg just START_LOG_SEGMENT(txid) and nothing else).

Okay, I am surprise to find this. All along, in previous discussions, I have been assuming that JournalManager calls roll to JournalService and the startLog transaction is recorded in JournalService. This is when epoch also gets persisted along with that record.

bq. I think it's just a matter of getting the ordering right. Before starting a log segment, you need to fence prior writers. The fencing step is what writes down the epoch. Then, when you create a new log segment, you tag it (eg by storing it in a directory per-epoch, or by writing a metadata file next to it before you create the file). I think this is sufficiently atomic.

Whether you store it in a directory per-epoch or record it in the startlogSegment record at the beginning of the segment - they are essentially the same.
                
> Persist the epoch received by the JournalService
> ------------------------------------------------
>
>                 Key: HDFS-3212
>                 URL: https://issues.apache.org/jira/browse/HDFS-3212
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, name-node
>    Affects Versions: Shared journals (HDFS-3092)
>            Reporter: Suresh Srinivas
>
> epoch received over JournalProtocol should be persisted by JournalService.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira