Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 30 Mar 2011 01:57:06 +0000 (UTC)
From: "Jitendra Nath Pandey (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1261775375.20477.1301450226313.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: <31416615.309351294823386532.JavaMail.jira@thor>
Subject: [jira] [Commented] (HDFS-1580) Add interface for generic Write
 Ahead Logging mechanisms
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012817#comment-13012817 ] 

Jitendra Nath Pandey commented on HDFS-1580:
--------------------------------------------

Ivan,
   I am thinking of something like following code. I am using name TxnLogOutputStream instead of EditLogOutputStream because the later confuses with existing EditLogOutputStream in Hdfs. This undoes some of my previous suggestions e.g. regarding putting transaction id in the URI. It seems better that the Journal is viewed as a single input or output stream and URI just identifies the namespace.
   I think we should use TxnRecord object which encapsulate a transaction as a byte-array instead of using Writables, its better to keep interface unaware of serialization protocol.  

{code}

abstract class JournalWriter {
     //The URI identifies the journal location & namespace
     JournalWriter(URI u);

     //Initializes the layoutVersion
     void format(int layoutVersion);

     //After this call, any subsequent transactions should
     //map to newLayoutVersion, to be used in upgrade or rollback.
     void setLayoutVersion(int newLayoutVersion);

     TxnLogOutputStream getOutputStream();

     //Deletes all transactions older than uptoTxnId (inclusive)
     void delete(long uptoTxnId);
}
class TxnRecord {
   long txnId;
   byte[] txn;
}
interface TxnLogOutputStream {
     write(long txnId, TxnRecord rec);
     void roll();
     close();
}
abstract class JournalReader {
     JournalReader(URI u);

     TxnLogInputStream getInputStream(long sinceTxnId);
}
interface TxnLogInputStream {
     TxnRecord next();

     //Returns layout version of the next record.
     int getLayoutVersion();
     void close();
}

{code}

> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>
>                 Key: HDFS-1580
>                 URL: https://issues.apache.org/jira/browse/HDFS-1580
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>         Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf, generic_wal_iface.pdf, generic_wal_iface.txt
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira