ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hurley (JIRA)" <>
Subject [jira] [Commented] (AMBARI-7221) Ambari Server REST API Memory Leak
Date Tue, 09 Sep 2014 14:51:29 GMT


Jonathan Hurley commented on AMBARI-7221:

The cause is a problem between Jetty and Spring Security where Spring is creating a new session
ID for every request. The default behavior of Spring is to create new sessions that can be
reused. The Jetty sessions were being created and never invalidated since the REST APIs are
stateless and don't really use sessions.

Removing session support seemed to be the most correct choice, especially since the agent
root was also configured to use sessions (even though they don't). In the future, if a type
of authentication model requires sessions, then we need to set IDLE timeouts so that Jetty
will reap the invalidated sessions.

> Ambari Server REST API Memory Leak
> ----------------------------------
>                 Key: AMBARI-7221
>                 URL:
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 1.6.0
>         Environment: Oracle JDK 1.7.0_45 on CentOS 6.5 64-bit
> SLES 11 SP1 64-bit with Oracle JDK 1.7 
>            Reporter: Jonathan Hurley
>            Assignee: Jonathan Hurley
>            Priority: Critical
>             Fix For: 1.7.0
> PROBLEM: When running many 1000's of REST API calls, the heap memory usage of the Ambari
server slowly goes up. Forcing a garbage collection will reclaim some of the heap but it never
goes back down to its starting level. Eventually the JVM is forced to do a full GC which hangs
the Ambari server for several minutes and, again, doesn't free all of the heap used by the
API calls.
> The Ambari server stopped working with out of heap space messages.
> STEPS TO REPRODUCE: I was able to reproduce the issue using this simple script run on
the Ambari server machine:
> {{while [ 1 ]; do curl -u admin:admin http://localhost:8080/api/v1/clusters; done}}
> ACTUAL BEHAVIOR: Ambari server heap memory usage increases over time when the REST API
is used repeatedly. GC does not free it.
> EXPECTED BEHAVIOR: Memory used by REST API connections should be released after a GC.

This message was sent by Atlassian JIRA

View raw message