Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Thu, 31 May 2012 14:14:23 +0000 (UTC)
From: "Robert Joseph Evans (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <1386243138.21983.1338473663548.JavaMail.jiratomcat@issues-vm>
In-Reply-To: <1838589788.14115.1338341123105.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Updated] (HADOOP-8452) DN logs backtrace when running under
 jsvc and /jmx is loaded
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-8452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Joseph Evans updated HADOOP-8452:
----------------------------------------

       Resolution: Fixed
    Fix Version/s: 3.0.0
                   2.0.1-alpha
           Status: Resolved  (was: Patch Available)

Thanks Andy,

I checked this into trunk and branch-2.
                
> DN logs backtrace when running under jsvc and /jmx is loaded
> ------------------------------------------------------------
>
>                 Key: HADOOP-8452
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8452
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Andy Isaacson
>            Assignee: Andy Isaacson
>            Priority: Minor
>             Fix For: 2.0.1-alpha, 3.0.0
>
>         Attachments: 95db1fa.txt, hadoop-8447-2.patch, hadoop-8447-2.patch, hadoop-8447.patch
>
>
> Running the data node under jsvc and requesting /jmx falls victim to a kernel bug http://marc.info/?l=linux-kernel&m=133788505209725&w=2 which results in EACCES when open()ing /proc/self/fd to attempt to count the open filedescriptors.
> Hopefully someday we will have kernels with this bug fixed; in the meantime, the log spew which results is unpleasant:
> 12270 2012-05-15 21:04:41,683 ERROR org.apache.hadoop.jmx.JMXJsonServlet: getting attribute OpenFileDescriptorCount of java.lang:type=OperatingSystem threw an exception
> 12271 javax.management.RuntimeErrorException: java.lang.InternalError: errno: 13 error: Unable to open directory /proc/self/fd
> 12272 
> 12273         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:858)
> 12274         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:869)
> 12275         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:670)
> 12276         at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638)
> 12277         at org.apache.hadoop.jmx.JMXJsonServlet.writeAttribute(JMXJsonServlet.java:314)
> 12278         at org.apache.hadoop.jmx.JMXJsonServlet.listBeans(JMXJsonServlet.java:292)
> 12279         at org.apache.hadoop.jmx.JMXJsonServlet.doGet(JMXJsonServlet.java:192)
> 12280         at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> 12281         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> 12282         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> 12283         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> 12284         at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:932)
> 12285         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> 12286         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> 12287         at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> 12288         at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> 12289         at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> 12290         at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> 12291         at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> 12292         at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> 12293         at org.mortbay.jetty.Server.handle(Server.java:326)
> 12294         at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> 12295         at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> 12296         at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> 12297         at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> 12298         at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> 12299         at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> 12300         at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> 12301 Caused by: java.lang.InternalError: errno: 13 error: Unable to open directory /proc/self/fd
> 12302 
> 12303         at com.sun.management.UnixOperatingSystem.getOpenFileDescriptorCount(Native Method)
> 12304         at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
> 12305         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 12306         at java.lang.reflect.Method.invoke(Method.java:597)
> 12307         at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:167)
> 12308         at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:96)
> 12309         at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:33)
> 12310         at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
> 12311         at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:65)
> 12312         at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:216)
> 12313         at javax.management.StandardMBean.getAttribute(StandardMBean.java:358)
> 12314         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666)
> 12315         ... 25 more
> we should catch the RuntimeErrorException and make it a debug().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira