aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Khutornenko <ma...@apache.org>
Subject Re: Review Request 44471: Implementing dbsnapshotting
Date Mon, 07 Mar 2016 23:55:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44471/
-----------------------------------------------------------

(Updated March 7, 2016, 11:55 p.m.)


Review request for Aurora, Bill Farner and Zameer Manji.


Changes
-------

Bill's comments.


Bugs: AURORA-1627
    https://issues.apache.org/jira/browse/AURORA-1627


Repository: aurora


Description (updated)
-------

H2 DB is now `script`ed into a dedicated `dbScript` field in `Snapshot` thrift struct to speed
up recovery under H2 MVStore.

There was extra care needed to ensure task stores are functioning properly when they are toggled
between in-memory and DB mode. Tried to document as much as possible. Let me know if something
is still unclear. The existing thrift snapshot fields will be supported until all stores are
fully migrated to H2 and removed subsequently along with in-memory store code.

`SnapshotStoreImplTest` got rewritten into `SnapshotStoreImplIT` to fully test `TaskStore`
migration matrix.

Also, moved checkstyle suppressions into annotations for more precise control.

Master:
```
Benchmark                                                   (updateCount)   Mode  Cnt  Score
  Error  Units
SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run              1  thrpt    5  4.876
± 0.403  ops/s
SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run              5  thrpt    5  0.942
± 0.032  ops/s
SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run             10  thrpt    5  0.436
± 0.040  ops/s
```

This patch:
```
Benchmark                                                   (updateCount)   Mode  Cnt  Score
  Error  Units
SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run              1  thrpt    5  9.846
± 0.886  ops/s
SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run              5  thrpt    5  1.977
± 0.319  ops/s
SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run             10  thrpt    5  1.310
± 0.139  ops/s
```


Diffs (updated)
-----

  api/src/main/thrift/org/apache/aurora/gen/storage.thrift 6dc46147bb0703e83a210a81ee24081183389a89

  config/checkstyle/checkstyle.xml 2074beb66ca19eb2da3d444bc64e583409e30ead 
  src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 50044e1c38c712a52855dd96fa2a9de769d6c973

  src/jmh/java/org/apache/aurora/benchmark/SnapshotBenchmarks.java ca484fab2a9b136c6d5b9be31e1ad1a5360f1b7a

  src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java 92849d9a82dcd6d5fe6cc452061bcdca31415f40

  src/main/java/org/apache/aurora/scheduler/storage/Storage.java 5124d17c9bc755e33a3e97b914caed34b77cdb94

  src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java 46b3d104931f175314902ccd39b81c4b6d67d4f3

  src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 6d8fa11d0aed841388e1e87fc0acdaf6b8f42d06

  src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java cca92dd6e9e52f9e38394c4c688ab4897eb8e23f

  src/main/java/org/apache/aurora/scheduler/storage/log/LogStorageModule.java ed63a7471d654dcefd2ff24e2e462974883541f2

  src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java db90150b48c5b134dde6c69f70ebca82bfdd0c12

  src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java 2f07afb73f6e585a6b43be68134e2beecac83d31

  src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java 172dd206d09f131807cb33fe841ca6ebc8198a14

  src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java PRE-CREATION

  src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplTest.java 44078675286dec908da7a404ae8fce7c495a7019

  src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 75130a315573dbb9eff1a9b1c5408df27623cf54


Diff: https://reviews.apache.org/r/44471/diff/


Testing
-------

./gradlew -Pq build
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh


Thanks,

Maxim Khutornenko


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message