hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2227) HDFS-2018 Part 2 : getRemoteEditLogManifest should pull it's information from FileJournalManager
Date Sat, 06 Aug 2011 00:08:26 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Todd Lipcon updated HDFS-2227:

    Attachment: hdfs-2227.txt

Took a look through your patch and made some changes. Here's a summary of what I changed:

- since initJournals() was now only being called from the constructor of FSEditLog, I just
moved it straight in there.
- I made getEditLogManifest synchronized, which I think was probably a bug before
- Tightened the try/catch around {{fj.getRemoteEditLog}} since that's the only line that can
actually throw - clearer to see scope of potential exceptions
- changed log message that would result there to include the exception trace
- added a precondition to ensure that we always find logs that start exactly at the given
transaction ID. Since we currently have that as an invariant, I prefer to make it explicit
everywhere rather than risk a bug where part of the system handles it and another doesn't.
- Because of above assumption, was able to simplify the code in FileJournalManager a bit too.
Since we don't use getLogFiles() anywhere else, I collapsed it with getRemoteEditLog.
- reverted your change to make NNStorage.format(StorageDirectory) public - it's not necessary,
we can use the already public format method from TestEditLog
- changed TXNS_PER_ROLL to 10 so that the filenames are 1-10, 11-20, 21-30, etc. Just makes
the test easier to look at :)
- added code to dump the contents of the edits directories to the logs in this test case -
made it easier to understand the tests for me.
- some refactoring of test utils to be used from multiple places
- added new unit test for FileJournalManager that tests the {{getRemoteEditLog}} method in
isolation with a mock directory

> HDFS-2018 Part 2 :  getRemoteEditLogManifest should pull it's information from FileJournalManager
> -------------------------------------------------------------------------------------------------
>                 Key: HDFS-2227
>                 URL: https://issues.apache.org/jira/browse/HDFS-2227
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-2227.diff, delta.txt, hdfs-2227.txt
> This is the second part of HDFS-2018. This patch moves the code that selects the available
RemoteEditLogManifest out of the transactional inspector and into FileJournalManager. 

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message