hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-326) Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.
Date Wed, 28 Apr 2010 21:34:01 GMT

     [ https://issues.apache.org/jira/browse/HDFS-326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steve Loughran updated HDFS-326:
--------------------------------

    Attachment: HDFS-326.patch

This is the patches for the -hdfs JAR to bring the NN and DN under the lifecycle patch supplied
with the latest version of HADOOP-6194. I'm not submitting it as a patch as it will only build
against that version; it's here for people to see. The tests all appear to pass on my machine.

This patch has a few interesting features
# Does not start the service in the constructor, to make subclassing it safer -the secondary
NN benefits from this
# Relies on the static factory methods to create and start instances of primary or secondary
NNs.
# Adds more shutdown logic.

The extra shutdown stuff could be separated off from the rest of the patch and applied first
-it would make this patch smaller.

> Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-326
>                 URL: https://issues.apache.org/jira/browse/HDFS-326
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: AbstractHadoopComponent.java, HADOOP-3628-18.patch, HADOOP-3628-19.patch,
hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch,
hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch,
hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch,
hadoop-3628.patch, hadoop-3628.patch, hadoop-lifecycle-tomw.sxw, hadoop-lifecycle.pdf, hadoop-lifecycle.pdf,
hadoop-lifecycle.sxw, HDFS-326.patch
>
>
> I'd like to propose we have a standard interface for hadoop components, the things that
get started or stopped when you bring up a namenode. currently, some of these classes have
a stop() or shutdown() method, with no standard name/interface, but no way of seeing if they
are live, checking their health of shutting them down reliably. Indeed, there is a tendency
for the spawned threads to not want to die; to require the entire process to be killed to
stop the workers. 
> Having a standard interface would make it easier for 
>  * management tools to manage the different things
>  * monitoring the state of things
>  * subclassing
> The latter is interesting as right now TaskTracker and JobTracker start up threads in
their constructor; that's very dangerous as subclasses may have their methods called before
they are full initialised. Adding this interface would be the right time to clean up the startup
process so that subclassing is less risky.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message