hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2227) HDFS-2018 Part 2 : getRemoteEditLogManifest should pull it's information from FileJournalManager
Date Sat, 06 Aug 2011 00:08:26 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Todd Lipcon updated HDFS-2227:
------------------------------

    Attachment: hdfs-2227.txt
                delta.txt

Took a look through your patch and made some changes. Here's a summary of what I changed:

- since initJournals() was now only being called from the constructor of FSEditLog, I just
moved it straight in there.
- I made getEditLogManifest synchronized, which I think was probably a bug before
- Tightened the try/catch around {{fj.getRemoteEditLog}} since that's the only line that can
actually throw - clearer to see scope of potential exceptions
- changed log message that would result there to include the exception trace
- added a precondition to ensure that we always find logs that start exactly at the given
transaction ID. Since we currently have that as an invariant, I prefer to make it explicit
everywhere rather than risk a bug where part of the system handles it and another doesn't.
- Because of above assumption, was able to simplify the code in FileJournalManager a bit too.
Since we don't use getLogFiles() anywhere else, I collapsed it with getRemoteEditLog.
- reverted your change to make NNStorage.format(StorageDirectory) public - it's not necessary,
we can use the already public format method from TestEditLog
- changed TXNS_PER_ROLL to 10 so that the filenames are 1-10, 11-20, 21-30, etc. Just makes
the test easier to look at :)
- added code to dump the contents of the edits directories to the logs in this test case -
made it easier to understand the tests for me.
- some refactoring of test utils to be used from multiple places
- added new unit test for FileJournalManager that tests the {{getRemoteEditLog}} method in
isolation with a mock directory

> HDFS-2018 Part 2 :  getRemoteEditLogManifest should pull it's information from FileJournalManager
> -------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-2227
>                 URL: https://issues.apache.org/jira/browse/HDFS-2227
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-2227.diff, delta.txt, hdfs-2227.txt
>
>
> This is the second part of HDFS-2018. This patch moves the code that selects the available
RemoteEditLogManifest out of the transactional inspector and into FileJournalManager. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message