aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Erb <s...@apache.org>
Subject Re: Review Request 52588: Enable per task volume mounts via scheduler API
Date Tue, 11 Oct 2016 16:50:14 GMT


> On Oct. 8, 2016, 9:40 a.m., Stephan Erb wrote:
> > src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql, lines 225-240
> > <https://reviews.apache.org/r/52588/diff/4/?file=1527281#file1527281line225>
> >
> >     I am new to the Aurora DB layer, so question ahead: Why do we need this _and_
the migration script? Is one of those genreated automatically from the other?
> 
> Zameer Manji wrote:
>     I did not work on our migration code, so I have no idea why we need both. I imagine
we could just replace schema.sql with the migrations, but I believe there is some nuance there
that is hard to capture. I will also defer to Josh here.
>     
>     One is not generated from the other sadly.
> 
> Stephan Erb wrote:
>     Thanks. Your response is already good enogh for now :)
> 
> Joshua Cohen wrote:
>     The gist of it is, when the scheduler starts up, we create the db from schema.sql.
After that, when we apply the snapshot from the replicated log, we check to see if there's
a DB script present. If so, we drop all tables from the DB and run the DML from that script.
That means, in the event that we're starting the scheduler after updating the version, that
we've now started up the scheduler with the DB matching the state it was from the previous
version. At that point we run the migrations on top of the old DB to apply any necessary schema
changes.
>     
>     In the event that the scheduler is started fresh with no snapshot (or the snapshot
does not contain a db script) then no migrations are run and the db is used as created from
schema.sql.
>     
>     Does that make sense?

So we are kind of vulnerable to bugs where our migration script does not exactly match the
provided default schema. We would never catch those bugs in Vagrant, as it is common to start
with a clean db there.

In systems like Django the initial schema is always fixed and the only way to change it is
through migrations. This goes beyond the scope of this RB but might be a direction to explore
in the future.


- Stephan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52588/#review151682
-----------------------------------------------------------


On Oct. 7, 2016, 9:19 p.m., Zameer Manji wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52588/
> -----------------------------------------------------------
> 
> (Updated Oct. 7, 2016, 9:19 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Santhosh Kumar Shanmugham, and Stephan Erb.
> 
> 
> Bugs: AURORA-1107
>     https://issues.apache.org/jira/browse/AURORA-1107
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This allows users to specify volume mounts for tasks using the unified
> containerizer if the operator permits them. This is analogous to enabling docker
> parameters per task and using the `--volume` parameter.
> 
> This does not include the needed DSL changes or an e2e test which will be in a
> subsequent diff.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md f3dd8bb0f983c560f29ac39824e517c9f145d69e 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 0c74665f750571ccfdeda5a0b71cccbe169716a7

>   docs/reference/scheduler-configuration.md e6b19f01ef276962143dabbaeec4fdf980291a28

>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java c6c2a6d4ec48681378210d1fcb7909fd088d2afb

>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 3bd22a0d47cf0793c749b008ccbdf58033c75a11

>   src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 701f79c900e1d5794741475333805d1c12af58dd

>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 9038c36e4ae05ade273a1c218b7a7e8af218ae39

>   src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java acb44984b4aa38bbe1cac8f805754d55dbe39f3a

>   src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java e137e57a0b111b9320e6902aa586d7f24e0d5c58

>   src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.java 151306a567448c45223e90c8a29ac6ecbc9f6653

>   src/main/java/org/apache/aurora/scheduler/storage/db/migration/V009_CreateContainerVolumesTable.java
PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java
e30c387239e868daedf496dd886df4d8c00f1fc3 
>   src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/VolumeModeTypeHandler.java
PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbContainer.java 8d4d7eca3a17a3b96bbbdec9271503e7e71b3aff

>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.xml 3fce25f5e6d180a20dc3ace83f666d47bf32c0c5

>   src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql e943c647af4d0bdb1c733813d019d91942fbbc95

>   src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
db9f276c1ad26b33ff66f679787ee798bbb69c80 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java bd052abd65109544be92d3e292f1c2b238c86135

>   src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 3d07292c83c229cc85d3ad624a7a7a65ef0dcd51

>   src/test/java/org/apache/aurora/scheduler/thrift/Fixtures.java 95b371627d23f7ed3472561f17880bf3c4259b96

>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java e578f5a3076b8f1aad86247b815d05c8244dddc8

> 
> Diff: https://reviews.apache.org/r/52588/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message