aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aurora ReviewBot <wfar...@apache.org>
Subject Re: Review Request 58036: Ensure enum tables are complete afer a snapshot restore.
Date Wed, 29 Mar 2017 19:32:32 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58036/#review170454
-----------------------------------------------------------



Master (c32f14c) is red with this patch.
  ./build-support/jenkins/build.sh

:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJavaNote: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh
:jsHint
:checkstyleMain[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java:49:8:
Unused import - org.apache.aurora.scheduler.storage.db.EnumBackfill. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:29:8:
Unused import - org.apache.aurora.gen.CronCollisionPolicy. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:30:8:
Unused import - org.apache.aurora.gen.JobUpdateAction. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:31:8:
Unused import - org.apache.aurora.gen.JobUpdateStatus. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:32:8:
Unused import - org.apache.aurora.gen.MaintenanceMode. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:33:8:
Unused import - org.apache.aurora.gen.Mode. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:34:8:
Unused import - org.apache.aurora.gen.ScheduleStatus. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:38:8:
Unused import - org.apache.aurora.scheduler.resources.ResourceType. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:1:
Line does not match expected header line of '^\/\*\*$'. [RegexpHeader]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:17:
First sentence should end with a period. [JavadocStyle]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:46:
Line is longer than 100 characters (found 102). [LineLength]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:54:
Line is longer than 100 characters (found 101). [LineLength]
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to
get more log output.

BUILD FAILED

Total time: 1 mins 24.435 secs


I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On March 29, 2017, 7:21 p.m., Zameer Manji wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58036/
> -----------------------------------------------------------
> 
> (Updated March 29, 2017, 7:21 p.m.)
> 
> 
> Review request for Aurora, Santhosh Kumar Shanmugham and Stephan Erb.
> 
> 
> Bugs: AURORA-1912
>     https://issues.apache.org/jira/browse/AURORA-1912
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> In our in memory database, we model enums as two column tables. The two columns
> would be `id` which corresponds to the integer value in the thrift enum and
> `name` which is the all caps string name of the enum. For example to model the
> `JobUpdateStatus` enum we have a table called `job_update_statuses`. In there
> the `ROLLING_FORWARD` enum is modeled as a row `(0, "ROLLING_FORWARD")`. Other
> tables reference the enum table via the id.
> 
> When we prepare storage on startup the `DbStorage` starts up. It does two
> things:
> 1. Load in the schema.
> 2. Populate the enum tables.
> 
> This ensures that when we insert values into the database, the enum refernces
> will be valid.
> 
> However, after we restore from a Snapshot with the `dbScript` field, we blow all
> of that data away and restore what was in the snapshot:
> ````
> try (Connection c = ((DataSource) store.getUnsafeStoreAccess()).getConnection()) {
>   LOG.info("Dropping all tables");
>   try (PreparedStatement drop = c.prepareStatement("DROP ALL OBJECTS")) \
>     drop.executeUpdate();
>   }
> ````
> 
> This means that if we add a new enum value, and then restore from a snapshot,
> that enum value will not exist in the table any more. We could address this by
> saying that every enum value addition requires a migration. However instead I
> propose not blowing away the work done by `DbStorage` instead and re-hydrating
> the enum tables.
> 
> To do this I extracted the logic into a new class `EnumBackfill`. Restoring from
> a snapshot calls this after the migrations are done. The underlying SQL was
> changed from `INSERT` to `MERGE` to make this work.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java f0b148cd158d61cd89cc51dca9f3fa4c6feb1b49

>   src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java 36a1bd5c784ed0febebccfd22e5064f0b2e3106f

>   src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java d1a196419b67108ee2bb778f83a2993e2e5ee83b

>   src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java 923e904f396724b9dde4a330ef312a6aae2c02a6

>   src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java PRE-CREATION

>   src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java 81a8cca6974e33c774473a4990e0e981cf6ddee6

>   src/main/resources/org/apache/aurora/scheduler/storage/db/EnumValueMapper.xml 153fd26c27275c46b190e71d8a5736153f2c2d18

>   src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java 42615da54096d5b06c7989cb30fc3cfbe59bc1b9

>   src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java f26529c76214c8f22563f04a197798c82d341b49

>   src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java ca9525665805a33b4a322a72022ff037f0dd2a94

> 
> 
> Diff: https://reviews.apache.org/r/58036/diff/1/
> 
> 
> Testing
> -------
> 
> existing tests and e2e tests
> 
> I also added a new enum value to `JobUpdateStatus` and observed it was correctly loaded
in.
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message