hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2223) Untangle depencencies between NN components
Date Wed, 03 Aug 2011 21:21:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13079034#comment-13079034
] 

Todd Lipcon commented on HDFS-2223:
-----------------------------------

First, here's my conception of what the responsibilities and interactions are meant to be:

*NameNode*: container class that handles startup/shutdown of the Java process itself. Starts/stops
RPC and HTTP interfaces.
*FSNamesystem*: container of both transient and persisted namespace state:
-- container for BlockManager, DatanodeManager, DelegationTokens, LeaseManager, etc
-- RPC calls that modify or inspect the namespace should get delegated here
--- anything that touches only blocks (eg block reports) should get delegated to BlockManager
--- anything that touches only file information (eg permissions, mkdirs) should get delegated
to FSDirectory
--- anything that crosses two of the above components should be coordinated here
-- should log mutations to FSEditLog
*FSDirectory*:
-- Namespace state that persists across restarts: the tree of inodes, and for each inode,
the block list
-- should log mutations to FSEditLog
*FSImage*: responsible for the mechanisms by which an FSNamesystem is snapshotted and logged
-- should be responsible for lifecycle of FSEditLog and NNStorage
*NNStorage*: should only be responsible for file management inside storage directories, and
tracking failures states of those directories
-- should not have any references "up" to other classes
*FSEditLog*: responsible for managing a set of journals
-- after initialization, should not need to reference any other classes

Hopefully this jives with what other people understand to be the "ideal" we're going for.

> Untangle depencencies between NN components
> -------------------------------------------
>
>                 Key: HDFS-2223
>                 URL: https://issues.apache.org/jira/browse/HDFS-2223
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>
> Working in the NN a lot for HA (HDFS-1623) I've come across a number of situations where
the tangled dependencies between NN components has been problematic for adding new features
and for testability. It would be good to untangle some of these and clarify what the distinction
is between the different components: NameNode, FSNamesystem, FSDirectory, FSImage, NNStorage,
and FSEditLog

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message