hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2018) Move all journal stream management code into one place
Date Fri, 08 Jul 2011 08:35:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13061844#comment-13061844
] 

Ivan Kelly commented on HDFS-2018:
----------------------------------

Good comments, I have a 3 hour flight later so I'll try to address them on it. A few points.
Firstly, I'd really like to get this into 0.23. Once this patch is in, bookkeeper support
can be added as a third party module with very little effort for 0.23, and it can be shipped
as a builtin in later releases.

The remoteEditLogManifest will only be used by file based journals. For bookkeeper the checkpointer
will simply be configured to use bookkeeper as the editlog. GetImageServlet will be taken
out of the equation for edits.

Regarding the startup behaviour, I would need to add getFirstTxId() and getLastTxId() members
to EditLogInputStream to be able to create a plan beforehand. Ill have a go at this later,
and try to submit another patch once I get back near an internet connection.

> Move all journal stream management code into one place
> ------------------------------------------------------
>
>                 Key: HDFS-2018
>                 URL: https://issues.apache.org/jira/browse/HDFS-2018
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: Edit log branch (HDFS-1073)
>
>         Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff
>
>
> Currently in the HDFS-1073 branch, the code for creating output streams is in FileJournalManager
and the code for input streams is in the inspectors. This change does a number of things.
>   - Input and Output streams are now created by the JournalManager.
>   - FSImageStorageInspectors now deals with URIs when referring to edit logs
>   - Recovery of inprogress logs is performed by counting the number of transactions instead
of looking at the length of the file.
> The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message