ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Wagle (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-2024) Ambari Server becomes unresponsive after crashing on http reads on jersey.
Date Thu, 25 Apr 2013 20:48:15 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Siddharth Wagle updated AMBARI-2024:
------------------------------------

    Description: 
The api's are being handled by a queuedthreadpool. The queuedthread pool size is 25.

Somehow the http connections are being torn down from the UI side but the server still is
hanging onto that socket and reading (most likely UI will also need to close http connections
if its not using them - which might be an issue as well but doesnt have to addressed as urgent).
The server has a read timeout of 0 which means it will just hang on to that socket for read.
This causes all the threads to block at one time or the other. Simple solution is add read
timeouts to all the SelectChannelConnector and SslSelectChannelConnector we use.

Exception trace: 

SEVERE: The exception contained within MappableContainerException could not be mapped to a
response, re-throwing to the HTTP container
org.eclipse.jetty.io.EofException: early EOF
        at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65)
        at org.codehaus.jackson.impl.ByteSourceBootstrapper.ensureLoaded(ByteSourceBootstrapper.java:507)
        at org.codehaus.jackson.impl.ByteSourceBootstrapper.detectEncoding(ByteSourceBootstrapper.java:129)
        at org.codehaus.jackson.impl.ByteSourceBootstrapper.constructParser(ByteSourceBootstrapper.java:224)
        at org.codehaus.jackson.JsonFactory._createJsonParser(JsonFactory.java:785)
        at org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:561)
        at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:414)
        at com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy.readFrom(JacksonProviderProxy.java:139)
        at com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)


Notice the API's is being called all the time - meaning they probalby had a browser up and
running for a long time.

There might be a possibilility that the browser might have some issues after running for a
long time. Something to keep in mind when this happens again. Easy way to check that is to
call Ambari server API's and also bring up a new browser window (new instance) and try hitting
the browser UI.

  was:
SEVERE: The exception contained within MappableContainerException could not be mapped to a
response, re-throwing to the HTTP container
org.eclipse.jetty.io.EofException: early EOF
        at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65)
        at org.codehaus.jackson.impl.ByteSourceBootstrapper.ensureLoaded(ByteSourceBootstrapper.java:507)
        at org.codehaus.jackson.impl.ByteSourceBootstrapper.detectEncoding(ByteSourceBootstrapper.java:129)
        at org.codehaus.jackson.impl.ByteSourceBootstrapper.constructParser(ByteSourceBootstrapper.java:224)
        at org.codehaus.jackson.JsonFactory._createJsonParser(JsonFactory.java:785)
        at org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:561)
        at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:414)
        at com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy.readFrom(JacksonProviderProxy.java:139)
        at com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)


Notice the API's is being called all the time - meaning they probalby had a browser up and
running for a long time.

There might be a possibilility that the browser might have some issues after running for a
long time. Something to keep in mind when this happens again. Easy way to check that is to
call AMbari server API's and also bring up a new browser window (new instance) and try hitting
the browser UI.

    
> Ambari Server becomes unresponsive after crashing on http reads on jersey.
> --------------------------------------------------------------------------
>
>                 Key: AMBARI-2024
>                 URL: https://issues.apache.org/jira/browse/AMBARI-2024
>             Project: Ambari
>          Issue Type: Bug
>          Components: controller
>    Affects Versions: 1.3.0
>            Reporter: Siddharth Wagle
>            Assignee: Siddharth Wagle
>             Fix For: 1.3.0
>
>         Attachments: AMBARI-2024.patch
>
>
> The api's are being handled by a queuedthreadpool. The queuedthread pool size is 25.
> Somehow the http connections are being torn down from the UI side but the server still
is hanging onto that socket and reading (most likely UI will also need to close http connections
if its not using them - which might be an issue as well but doesnt have to addressed as urgent).
The server has a read timeout of 0 which means it will just hang on to that socket for read.
This causes all the threads to block at one time or the other. Simple solution is add read
timeouts to all the SelectChannelConnector and SslSelectChannelConnector we use.
> Exception trace: 
> SEVERE: The exception contained within MappableContainerException could not be mapped
to a response, re-throwing to the HTTP container
> org.eclipse.jetty.io.EofException: early EOF
>         at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65)
>         at org.codehaus.jackson.impl.ByteSourceBootstrapper.ensureLoaded(ByteSourceBootstrapper.java:507)
>         at org.codehaus.jackson.impl.ByteSourceBootstrapper.detectEncoding(ByteSourceBootstrapper.java:129)
>         at org.codehaus.jackson.impl.ByteSourceBootstrapper.constructParser(ByteSourceBootstrapper.java:224)
>         at org.codehaus.jackson.JsonFactory._createJsonParser(JsonFactory.java:785)
>         at org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:561)
>         at org.codehaus.jackson.jaxrs.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:414)
>         at com.sun.jersey.json.impl.provider.entity.JacksonProviderProxy.readFrom(JacksonProviderProxy.java:139)
>         at com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)
> Notice the API's is being called all the time - meaning they probalby had a browser up
and running for a long time.
> There might be a possibilility that the browser might have some issues after running
for a long time. Something to keep in mind when this happens again. Easy way to check that
is to call Ambari server API's and also bring up a new browser window (new instance) and try
hitting the browser UI.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message