aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maxim Khutornenko" <ma...@apache.org>
Subject Re: Review Request 26232: Implementing update history pruner.
Date Wed, 01 Oct 2014 19:18:27 GMT


> On Oct. 1, 2014, 5:42 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java, line 161
> > <https://reviews.apache.org/r/26232/diff/1/?file=710137#file710137line161>
> >
> >     We also need time-based retention.  How about 1 month by default?
> 
> Maxim Khutornenko wrote:
>     Thought about that but was not sure the extra complexity is worth it. Also, don't
we want to show update history in the UI regardless of how old it is?
> 
> Bill Farner wrote:
>     Given that it currently needs to fit in memory, i think not.
>     
>     For example: if i create a job to test something out, then kill it and never return.
 Probably shouldn't retain those updates forever.
> 
> David McLaughlin wrote:
>     Wouldn't the updates in that case be killed by the job garbage collection/cascading
deletion of the job key?
> 
> Bill Farner wrote:
>     If so, that warrants validation in an integration test between TaskHistoryPruner,
storage, and DbJobUpdateStore.

We don't cascade deletes into the job_keys table. In fact, we don't actually delete from job_keys
at all at this point. We should probably address this when converting TaskStore into H2.

| if i create a job to test something out, then kill it and never return.  Probably shouldn't
retain those updates forever.
Good point. This alone warrants adding time-based cleanup.


- Maxim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26232/#review55088
-----------------------------------------------------------


On Oct. 1, 2014, 5:26 p.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26232/
> -----------------------------------------------------------
> 
> (Updated Oct. 1, 2014, 5:26 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Bill Farner.
> 
> 
> Bugs: AURORA-743
>     https://issues.apache.org/jira/browse/AURORA-743
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> The pruner runs on periodic basis and trims completed updates up to the guranteed per
job retention count.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java aa45d27391b1786ca3d5e8c928045f1b6f3cf5ef

>   src/main/java/org/apache/aurora/scheduler/async/HistoryPruner.java ebae58a04e8857c5f26d4b57c27dfcda9e14c82c

>   src/main/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPruner.java PRE-CREATION

>   src/main/java/org/apache/aurora/scheduler/storage/JobUpdateStore.java c3abffe575e801cebec3572cf4aceac83a238b55

>   src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java 3db0114c5432a5b7d0b01d97c75494be9e3e99a3

>   src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java c583e085e0458835d51ebf740a3b5f01b428bb25

>   src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java 66c91644677e7176ccf53dcfcf29a6792ec398bc

>   src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml
631ab2543dadb77b7fd1cae96adc9c0bd2f94b39 
>   src/test/java/org/apache/aurora/scheduler/async/HistoryPrunerTest.java 011d9ec5d84c658c2b2e39bb0b3f9d20b5440cee

>   src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java PRE-CREATION

>   src/test/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStoreTest.java 1b1160f349d64bbcd4d20103a82f3b8eb1ca75d9

> 
> Diff: https://reviews.apache.org/r/26232/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message