hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2018) 1073: Move all journal stream management code into one place
Date Mon, 15 Aug 2011 21:59:28 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085361#comment-13085361
] 

Jitendra Nath Pandey commented on HDFS-2018:
--------------------------------------------

Todd,
  I think there are two fundamental differences in the two patches. 

1) EditLogReference or TransactionRanges in the API
   >I found the code to implement this "merge distinct sets of ranges to cover the target
range" algorithm a lot easier to understand when it's dealing 
   > with the full List<EditLogReference>, rather than having to keeping going back
to each JournalManager in turn as it builds up a list. 
   In HDFS-2158, I intend to simplify selectInputStream. JournalSet will hide away all the
journals and FSEditLog will just know one journal. There is a patch uploaded on 2158. It is
a bit old but will give you the idea. Will that address your concern?

2) Ivan's patch attempts to hide log recovery from FSEditLog, however your patch makes it
explicit.
    Can this be addressed in a separate jira, in the context of HA, because we run into the
case of concurrent reader/writer only in the context of HA?


> 1073: Move all journal stream management code into one place
> ------------------------------------------------------------
>
>                 Key: HDFS-2018
>                 URL: https://issues.apache.org/jira/browse/HDFS-2018
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 0.23.0
>
>         Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
hdfs-2018-otherapi.txt, hdfs-2018.txt
>
>
> Currently in the HDFS-1073 branch, the code for creating output streams is in FileJournalManager
and the code for input streams is in the inspectors. This change does a number of things.
>   - Input and Output streams are now created by the JournalManager.
>   - FSImageStorageInspectors now deals with URIs when referring to edit logs
>   - Recovery of inprogress logs is performed by counting the number of transactions instead
of looking at the length of the file.
> The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message