Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <2824247.8511272490918759.JavaMail.jira@thor>
Date: Wed, 28 Apr 2010 17:41:58 -0400 (EDT)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-326) Add a lifecycle interface for Hadoop
 components: namenodes, job clients, etc.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861952#action_12861952 ] 

Steve Loughran commented on HDFS-326:
-------------------------------------

One test fails, TestFsck, on OS/X; I'm not sure if it is related to this patch or not. Thoughts?
{code}
2010-04-28 22:14:36,824 INFO  security.Groups (Groups.java:<init>(60)) - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000
2010-04-28 22:14:36,828 DEBUG security.UserGroupInformation (FSPermissionChecker.java:checkPermission(115)) - ACCESS CHECK: org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker@a01ec0b, doCheckOwner=false, ancestorAccess=null, parentAccess=null, access=null, subAccess=null
2010-04-28 22:14:36,829 DEBUG security.UserGroupInformation (FSPermissionChecker.java:checkPermission(115)) - ACCESS CHECK: org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker@42fcb4f, doCheckOwner=false, ancestorAccess=null, parentAccess=null, access=READ_EXECUTE, subAccess=null
2010-04-28 22:14:36,830 WARN  namenode.NameNode (NamenodeFsck.java:fsck(180)) - Fsck on path '/dfsck' FAILED
org.apache.hadoop.security.AccessControlException: Permission denied: user=ProbablyNotARealUserName, access=READ_EXECUTE, inode="dfsck":slo:supergroup:rwx------
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:207)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:142)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4016)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPathAccess(FSNamesystem.java:3980)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListing(FSNamesystem.java:2244)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.getListing(NameNode.java:937)
	at org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.check(NamenodeFsck.java:250)
	at org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.fsck(NamenodeFsck.java:159)
	at org.apache.hadoop.hdfs.server.namenode.FsckServlet$1.run(FsckServlet.java:65)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:738)
	at org.apache.hadoop.hdfs.server.namenode.FsckServlet.doGet(FsckServlet.java:53)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
	at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:760)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Permission denied: user=ProbablyNotARealUserName, access=READ_EXECUTE, inode="dfsck":slo:supergroup:rwx------


Fsck on path '/dfsck' FAILED
{code}


> Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-326
>                 URL: https://issues.apache.org/jira/browse/HDFS-326
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: AbstractHadoopComponent.java, HADOOP-3628-18.patch, HADOOP-3628-19.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-3628.patch, hadoop-lifecycle-tomw.sxw, hadoop-lifecycle.pdf, hadoop-lifecycle.pdf, hadoop-lifecycle.sxw, HDFS-326.patch
>
>
> I'd like to propose we have a standard interface for hadoop components, the things that get started or stopped when you bring up a namenode. currently, some of these classes have a stop() or shutdown() method, with no standard name/interface, but no way of seeing if they are live, checking their health of shutting them down reliably. Indeed, there is a tendency for the spawned threads to not want to die; to require the entire process to be killed to stop the workers. 
> Having a standard interface would make it easier for 
>  * management tools to manage the different things
>  * monitoring the state of things
>  * subclassing
> The latter is interesting as right now TaskTracker and JobTracker start up threads in their constructor; that's very dangerous as subclasses may have their methods called before they are full initialised. Adding this interface would be the right time to clean up the startup process so that subclassing is less risky.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.