Mailing-List: contact reviews-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: reviews@aurora.apache.org
Content-Type: multipart/alternative;
 boundary="===============5400296139269274219=="
MIME-Version: 1.0
Subject: Re: Review Request 60942: Remove task level resource fields from
 thrift interface and db
From: =?utf-8?q?Nicol=C3=A1s_Donatucci?= <ndonatucci@medallia.com>
To: Joshua Cohen <jcohen@apache.org>, Stephan Erb <serb@apache.org>,
 Zameer Manji <zmanji@apache.org>
Cc: =?utf-8?q?Nicol=C3=A1s_Donatucci?= <ndonatucci@medallia.com>,
 Aurora ReviewBot <wfarner@apache.org>, Aurora <reviews@aurora.apache.org>
Date: Mon, 07 Aug 2017 14:25:32 -0000
Message-ID: <20170807142532.44139.80570@reviews-vm2.apache.org>
Auto-Submitted: auto-generated
Sender: =?utf-8?q?Nicol=C3=A1s_Donatucci_=3Cnoreply=40reviews=2Eapache=2Eor?=@reviews.apache.org,
	=?utf-8?q?g=3E?=@reviews.apache.org
References: <20170804140427.17965.17566@reviews-vm2.apache.org>
In-Reply-To: <20170804140427.17965.17566@reviews-vm2.apache.org>
Reply-To: =?utf-8?q?Nicol=C3=A1s_Donatucci_=3Cndonatucci=40medallia=2Ecom?=@reviews.apache.org,
	=?utf-8?q?=3E?=@reviews.apache.org
archived-at: Mon, 07 Aug 2017 14:25:45 -0000

--===============5400296139269274219==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60942/
-----------------------------------------------------------

(Updated Aug. 7, 2017, 2:25 p.m.)


Review request for Aurora, Joshua Cohen, Stephan Erb, and Zameer Manji.


Changes
-------

Fixed broken client test. Thanks Stephan for the tip on how to do it!


Bugs: AURORA-1707
    https://issues.apache.org/jira/browse/AURORA-1707


Repository: aurora


Description
-------

Removed task level resource fields from the DB and the thrift-API.
To do this, a new DB migration was added. When upgrading, it just drops the task level resource fields. When downgrading, it creates the fields again and populates them with information from the task_resource table.

IMPORTANT: One of the python client tests is failing (test_cron_diff). This is not serious, I think it is a problem with the ordering of the elements of Resources (had similar problems with other python client tests that instead of comparing the Resources as a set, compared them as lists and thus order mattered). Nevertheless, I could not fully understand the code of that test. I was hoping someone could give me a hand with that. 
But then again, it is a smaller issue and so the patch can start being reviewed.

Issue Related: AURORA-1707


Diffs (updated)
-----

  api/src/main/thrift/org/apache/aurora/gen/api.thrift 3749531b5412d7ca217736aa85eed8e6606225ad 
  src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 186fa1b3a4780c0536fb486d50a33133258110cd 
  src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java d2eb6aa6e4a155b2d28debab2ca10dfc76d57213 
  src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.java cda55c55680a19ed421299a8949299b21949787b 
  src/main/java/org/apache/aurora/scheduler/storage/db/migration/V004_CreateTaskResourceTable.java af106a8a9ee8c14122e98bcc0ec44b616f21d61f 
  src/main/java/org/apache/aurora/scheduler/storage/db/migration/V011_DropResourceFields.java PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/storage/db/views/DbTaskConfig.java 138cd5316adc73eed24fc7accc53885dd5d5bee5 
  src/main/python/apache/aurora/client/cli/diff_formatter.py 78717774aa3fbaf83a5fb850bc9f9f4a4038d70f 
  src/main/python/apache/aurora/client/cli/jobs.py b79ae56bee0e5692cacf1e66f4a4126b06aaffdc 
  src/main/resources/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.xml 5422183e4a1fe122fc0e1aa871aa75ae102e322d 
  src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 7a86f47af67adb3e488381d30ddf424549deefbc 
  src/test/java/org/apache/aurora/scheduler/http/TestUtils.java 689482c9f6c49bcca781834566edeb975d2f3af2 
  src/test/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollectorTest.java caaba9b6dff46ff0b037759f1c817a321ae15ee4 
  src/test/java/org/apache/aurora/scheduler/thrift/Fixtures.java 43e32eede27bbf26363a3fd1ca34ffe6f8c01a73 
  src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 1bc2a778ad3f1543a055023f9ec3fe9e4a9523e3 
  src/test/python/apache/aurora/client/cli/test_status.py b0b7f96d8148f0dd1f6f45d1c5c6f830cabcfd5d 
  src/test/python/apache/aurora/client/cli/util.py 43db828ca1cccd91c88f016b4c994fef33182fbf 
  src/test/python/apache/aurora/config/test_resources.py 25a20f0b702189744a26b85053db34c37ff5b03c 


Diff: https://reviews.apache.org/r/60942/diff/4/

Changes: https://reviews.apache.org/r/60942/diff/3-4/


Testing
-------

src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh

1 - Created a new VM with a fresh Aurora 0.18.0, ran the tests so that the db would have something. Then upgraded to patched version and ran the tests again. Everything worked and task level resource fields were no longer there.
2 - Created a new VM with a fresh patched version, ran the tests so that the db would have something. Then downgraded to 0.18.0 and ran the tests again. Everything worked and task level resources were there, with the correct backfilled values.


Thanks,

Nicolás Donatucci


--===============5400296139269274219==--