aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David McLaughlin <da...@dmclaughlin.com>
Subject Re: Review Request 54774: Avoid double writing job updates to the Scheduler Snapshot
Date Thu, 15 Dec 2016 19:22:17 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54774/
-----------------------------------------------------------

(Updated Dec. 15, 2016, 7:22 p.m.)


Review request for Aurora, Joshua Cohen, Santhosh Kumar Shanmugham, and Zameer Manji.


Changes
-------

Fix the update logic. Add release notes.


Bugs: AURORA-1861
    https://issues.apache.org/jira/browse/AURORA-1861


Repository: aurora


Description
-------

Motivation: Thanks to the mybatis query metrics we added, we found that double writing Snapshot
fields for H2 stores adds considerable overhead to our snapshot creation time. 

Snapshots are also written as backups, and many operators choose to process backups offline
for analytics, rather than query the live scheduler (due to not being able to scale reads
horizontally). So this allows operators to enable/disable the hydrated fields as needed.


Diffs (updated)
-----

  RELEASE-NOTES.md 90c4793b218f54ad92ab4da5049d5e92c3a104b3 
  src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java 3fa408e283f91b313633959ea6d2e730d4dc0771

  src/main/java/org/apache/aurora/scheduler/storage/log/LogStorageModule.java 7dcd1bfcf303cf374e9a6627cb6c632ccea098f2

  src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java 853780bc68400489578ed3692920aafcec42c999

  src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java 7a11850e217dcb0148e4a4d33542c95b2e53a726

  src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java cf0a8f3ea11e9c48d1f16441af54dc781b33bdfc


Diff: https://reviews.apache.org/r/54774/diff/


Testing
-------

./gradlew test

I'll apply this to one of our test clusters before merging to master too.


Thanks,

David McLaughlin


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message