Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 4 Feb 2011 21:39:30 +0000 (UTC)
From: "Jitendra Nath Pandey (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <959923475.642.1296855570686.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: <31416615.309351294823386532.JavaMail.jira@thor>
Subject: [jira] Commented: (HDFS-1580) Add interface for generic Write Ahead
 Logging mechanisms
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990760#comment-12990760 ] 

Jitendra Nath Pandey commented on HDFS-1580:
--------------------------------------------

> > ShouldWALStreamCustodians control rolling themselves?
  As I understand, book-keeper doesn't allow reading logs from open ledger, is that correct? If that is the case it would be better to first roll and then do the checkpoint to capture most latest edits. 

>WALStreamCustodianNotifier
  It seems to me this will be implemented only by fsedit log. If that is the case we could handle errors just by exceptions i.e. if an operation on WALStreamCustodian interface fails an exception is thrown and fsedit log can decide to remove the custodian depending on the kind of exception.
 

> namenode crash
  If the namenode comes back before the znode disappears from zookeeper, the ledger will be open. In that case, will the FSEditLog.load be able to load transactions from the open ledger as well?

> interface JournalStream
  The document defines this interface but doesn't describe its purpose or use-case.

> List<URI> getLogs(long sinceTransactionId);
   The list returned must be ordered w.r.t the transactions contained. It might be a good idea to encode the ordering attribute in the url itself, so that the caller of this method can also verify that order is correct. The uri naming convention could mimic this aspect from the convention proposed in 1073.

> void startRoll()
> void endRoll()
    I can only imagine a single roll method, that cuts a log, and starts a new one. I believe the naming convention or the ordering attribute for the logs should be controlled by the application and not the storage, therefore the roll method should take a parameter which becomes part of the log metadata and is used to order the logs. Again this also depends on how 1073 does it for file logs.


> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>
>                 Key: HDFS-1580
>                 URL: https://issues.apache.org/jira/browse/HDFS-1580
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ivan Kelly
>         Attachments: generic_wal_iface.pdf, generic_wal_iface.pdf, generic_wal_iface.txt
>
>


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira