hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2018) Move all journal stream management code into one place
Date Thu, 16 Jun 2011 15:17:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050486#comment-13050486

Ivan Kelly commented on HDFS-2018:

Yes, I've been mulling over this in the back of my head of the last week, but haven't had
a chance to code it yet. I think what I need to do here is create a new method of FSEditLog
public class FSEditLog {
  JournalManager getBestJournal() {
    if (/* pre transaction version */) {
       FSImageOldStorageInspector inspector = new FSImageOldStorageInspector();
       // inspect all storagedirs in this.storage
       return new FilePreTransactionJournalManager(inspector.getEditLogFiles());
    } else {
       // choose by looping through edit log URIs, creating FileJournalManager objects
       // and returning the one with most transactions.

The creation of the FileJournalManager object can be outsourced to another method. It eventually
will anyhow, as the factory stuff will need ot be there.

> Move all journal stream management code into one place
> ------------------------------------------------------
>                 Key: HDFS-2018
>                 URL: https://issues.apache.org/jira/browse/HDFS-2018
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: Edit log branch (HDFS-1073)
>         Attachments: HDFS-2018.diff, HDFS-2018.diff
> Currently in the HDFS-1073 branch, the code for creating output streams is in FileJournalManager
and the code for input streams is in the inspectors. This change does a number of things.
>   - Input and Output streams are now created by the JournalManager.
>   - FSImageStorageInspectors now deals with URIs when referring to edit logs
>   - Recovery of inprogress logs is performed by counting the number of transactions instead
of looking at the length of the file.
> The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message