ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hurley" <jhur...@hortonworks.com>
Subject Review Request 30599: Ambari StageDAO.findByCommandStatuses causes Postgress HIGH CPU
Date Wed, 04 Feb 2015 01:18:34 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30599/
-----------------------------------------------------------

Review request for Ambari, Nate Cole and Tom Beerbower.


Bugs: AMBARI-9334
    https://issues.apache.org/jira/browse/AMBARI-9334


Repository: ambari


Description
-------

StageDAO.findByCommandStatuses performs a cartesion product between host_role_command and
stage in order to determine if there are any stages that are in progress (ie at least one
of the stages's commands is in progress). This was causing a major performance issue on Postgres,
especially since it's run on a thread every x seconds.

Because stage uses a compound primary key, there is no simple way to extract a DISTINCT set
of stages without doing the JOIN. It's also not feasible to simple request the commands and
extract the stage off of them for the following reason:
- There could be 1000's of commands in a large cluster
- The HostRoleCommandEntity binds lazily to its StageEntity

Adding a clustered index did increase performace by a decent amount on Postgres, but oddly
enough not on MySQL. I believe this is due to the fact that on MySQL the primary key counts
as the clustered index already and a table can only have 1 clustered index.

At the end of the day, my solution does the following:
- Checks to see if any commands are in progress; if there are none, don't even try the stage
lookup.
- If there are commands that are in progress, use a slighly more efficient JPQL query that
optimizes the compiled SQL


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java
1ef161a 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
4db1496 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java c8ed235

  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
9cc1075 
  ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleStatus.java 4d38b06

  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java dca8808

  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/StageDAO.java 8b1cfb3 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
8e08727 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java 058fe9b

  ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
3c80b4e 

Diff: https://reviews.apache.org/r/30599/diff/


Testing
-------

Some new tests written to cover the cases where there are many stages and many commands.

mvn clean test


Thanks,

Jonathan Hurley


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message