aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Erb <s...@apache.org>
Subject Re: Review Request 57757: Make Thermos observer resource collection intervals configurable
Date Sun, 19 Mar 2017 22:50:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57757/
-----------------------------------------------------------

(Updated March 19, 2017, 11:50 p.m.)


Review request for Aurora, Santhosh Kumar Shanmugham and Zameer Manji.


Summary (updated)
-----------------

Make Thermos observer resource collection intervals configurable


Bugs: AURORA-1907
    https://issues.apache.org/jira/browse/AURORA-1907


Repository: aurora


Description (updated)
-------

We have noticed that on hosts with lots of active tasks (~100) the observer UI
is not usable. Thermos fully utilizes one core but does not render any requests.

Dumping `/threads` indicates the observer might be backlogged by the hundred
concurrent `TaskResourceMonitor` threads. Due to the Python GIL only one can
make progress at a time though.

This patch is now adding options to control the resource collection interval,
giving operators a possibility to reduce the CPU pressure.

-- 

Reviewer notes: I dropped the entrypoint test as it was heavily relying on mocks
without significant code coverage.


Diffs
-----

  RELEASE-NOTES.md d58d2bd2558c2060aa659402ea88271bd8704cde 
  docs/README.md 1d679e27cf26122c34a9157194f189ce3035b176 
  docs/reference/observer-configuration.md PRE-CREATION 
  src/main/python/apache/aurora/tools/thermos_observer.py 4bba01930434f33150b0413d20d9657b61b364c5

  src/main/python/apache/thermos/monitoring/resource.py 53d0ff1a71c27f053c59acca556c35d1e5ac91f0

  src/main/python/apache/thermos/observer/task_observer.py 1485de8faef52716f11b82a3556064de26c67427

  src/test/python/apache/aurora/tools/test_thermos_observer_entry_point.py e1c8dec5f26be1aec931b1a614ecc138ae63d9de



Diff: https://reviews.apache.org/r/57757/diff/2/


Testing
-------

./pants test.pytest src/{test,main}/python:: -- -v


Thanks,

Stephan Erb


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message