hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2223) Untangle depencencies between NN components
Date Wed, 31 Aug 2011 23:33:10 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Todd Lipcon updated HDFS-2223:
------------------------------

    Attachment: hdfs-2223-7.txt

bq. FSImage creating FSNamesystem() does not seem right. Especially when FSNamesystem#loadFSImage()
calls FSImage#format(), which then creates a new FSNamesystem. It would be good to create
FSNamsystem and pass it to FSImage in #format().

Fixed. I had to also move the initialization of NNResourceChecker, since with this modification
FSNamesystem was being created before FSImage. NNResourceChecker should probably move down
into FSImage at some point, but I didn't want to broaden the scope of this JIRA further.

bq. BackupImage#journal() BackupImage#convergeJournalSpool() could take FSNamesystem as additional
argument. We could remove FSNamesystem member variable. BTW I do not see any call to BackupImage#saveCheckPoint().
These changes can be done in a separate jira.

Agreed this could be done separately.

bq. Question unrelated to this patch: why is the expectation that configuration have both
DFS_NAMENODE_CHECKPOINT_EDITS_DIR_KEY and DFS_NAMENODE_CHECKPOINT_DIR_KEY set, in FSImage#doImportCheckpoint()?

The {{dfs.namenode.checkpoint.edits.dir}} configuration defaults to {{${dfs.namenode.checkpoint.dir}}},
so setting just the former without explicitly setting the latter is sufficient.

bq. Please add javadoc to FSNamesystem#loadFromDisk()

Done

bq. Minor: FSNamesystem#initialize() - systemStart variable is no longer used

It's still used from getStartTime() and SafeMode.leave.

bq. Minor: Unnecessary change in NNStorage.java that pushes code beyond 80 columns
Oops, fixed.


> Untangle depencencies between NN components
> -------------------------------------------
>
>                 Key: HDFS-2223
>                 URL: https://issues.apache.org/jira/browse/HDFS-2223
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-2223-1.txt, hdfs-2223-2.txt, hdfs-2223-3.txt, hdfs-2223-4.txt,
hdfs-2223-5.txt, hdfs-2223-6.txt, hdfs-2223-7.txt
>
>
> Working in the NN a lot for HA (HDFS-1623) I've come across a number of situations where
the tangled dependencies between NN components has been problematic for adding new features
and for testability. It would be good to untangle some of these and clarify what the distinction
is between the different components: NameNode, FSNamesystem, FSDirectory, FSImage, NNStorage,
and FSEditLog

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message