hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
Date Mon, 21 Mar 2011 18:57:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009291#comment-13009291
] 

Todd Lipcon commented on HDFS-1580:
-----------------------------------

I'm not sure I agree with your point that rolling should be a concern of the JournalManager
and not of the higher-level frameworks. In HDFS-1073, Sanjay and I had come to the conclusion
that we should maintain the invariant that different edits logs should roll at the same time.
This makes it possible to take two edits directories (even across the BN and NN) and the files
inside should be md5sum-identical to each other -- very important for ops and peace of mind.

Perhaps some WAL implementions may want to ignore the roll boundaries, but we at least need
coordination between different File logs.

> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>
>                 Key: HDFS-1580
>                 URL: https://issues.apache.org/jira/browse/HDFS-1580
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>         Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf,
generic_wal_iface.txt
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message