hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
Date Mon, 21 Mar 2011 14:25:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009138#comment-13009138
] 

Ivan Kelly commented on HDFS-1580:
----------------------------------

This depends on whether we want the roll operation exposed to the user of the JournalManager.
In the past this was required as roll had to happen to allow a checkpoint to take place. However,
once HDFS-1580 goes in, its not strictly necessary because rolling is stateless. The JournalManager
could roll in the background without the user of the class having any knowledge of the roll.
To the user, the JournalManager is simply something to open, write to and close. Do you think
this simplification is worthwhile? 

I agree that getOutputStream(txid) everywhere is a bit messy. It also doesnt work. If a roll
happens while the OutputStream has been given out the editlog may end up writing to two different
streams which could mess up transaction ids. This isn't in any way unsolvable, but I need
to think about it more.

> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>
>                 Key: HDFS-1580
>                 URL: https://issues.apache.org/jira/browse/HDFS-1580
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>         Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf,
generic_wal_iface.txt
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message