ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nate Cole" <nc...@hortonworks.com>
Subject Re: Review Request 31683: RU: query for UpgradeItem is taking too long on large upgrades
Date Tue, 03 Mar 2015 15:45:19 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31683/
-----------------------------------------------------------

(Updated March 3, 2015, 10:45 a.m.)


Review request for Ambari, Jonathan Hurley and Tom Beerbower.


Bugs: AMBARI-9897
    https://issues.apache.org/jira/browse/AMBARI-9897


Repository: ambari


Description
-------

* Added a "Summary DTO" that will get all the stages' task statuses in one call to the DB,
indexed by stage.  This is used instead of iterating, and re-iterating several times over
stages/tasks for a request.
* Most notable, on large clusters we end up with 1000s of stages and tasks.  All that iterating
was consuming up to 2 minutes for the full upgrade/upgrade_group/upgrade_item/task tree. 
It's now down to about 5s (not great, but I'll take it).


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java
3a84529 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java
e55897c 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeGroupResourceProvider.java
a0ff3d9 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
9a56e2b 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java a55f383

  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandStatusSummaryDTO.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 76df049 
  ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java
6252d4e 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StageResourceProviderTest.java
1d80f47 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
ef4bd82 

Diff: https://reviews.apache.org/r/31683/diff/


Testing (updated)
-------

Manual testing of cluster install + upgrade to make sure both RU-related and non-RU-related
tasks showed the correct numbers.

Tests run: 2763, Failures: 0, Errors: 0, Skipped: 15

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21:34.415s
[INFO] Finished at: Tue Mar 03 10:41:42 EST 2015
[INFO] Final Memory: 32M/337M
[INFO] ------------------------------------------------------------------------


Thanks,

Nate Cole


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message