hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
Date Wed, 30 Mar 2011 01:57:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012817#comment-13012817

Jitendra Nath Pandey commented on HDFS-1580:

   I am thinking of something like following code. I am using name TxnLogOutputStream instead
of EditLogOutputStream because the later confuses with existing EditLogOutputStream in Hdfs.
This undoes some of my previous suggestions e.g. regarding putting transaction id in the URI.
It seems better that the Journal is viewed as a single input or output stream and URI just
identifies the namespace.
   I think we should use TxnRecord object which encapsulate a transaction as a byte-array
instead of using Writables, its better to keep interface unaware of serialization protocol.


abstract class JournalWriter {
     //The URI identifies the journal location & namespace
     JournalWriter(URI u);

     //Initializes the layoutVersion
     void format(int layoutVersion);

     //After this call, any subsequent transactions should
     //map to newLayoutVersion, to be used in upgrade or rollback.
     void setLayoutVersion(int newLayoutVersion);

     TxnLogOutputStream getOutputStream();

     //Deletes all transactions older than uptoTxnId (inclusive)
     void delete(long uptoTxnId);
class TxnRecord {
   long txnId;
   byte[] txn;
interface TxnLogOutputStream {
     write(long txnId, TxnRecord rec);
     void roll();
abstract class JournalReader {
     JournalReader(URI u);

     TxnLogInputStream getInputStream(long sinceTxnId);
interface TxnLogInputStream {
     TxnRecord next();

     //Returns layout version of the next record.
     int getLayoutVersion();
     void close();


> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>                 Key: HDFS-1580
>                 URL: https://issues.apache.org/jira/browse/HDFS-1580
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>         Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf,
generic_wal_iface.pdf, generic_wal_iface.txt

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message