aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David McLaughlin <da...@dmclaughlin.com>
Subject Re: Review Request 59699: Improve task history pruning by batch deleting tasks
Date Thu, 01 Jun 2017 21:37:50 GMT


> On June 1, 2017, 9:22 p.m., David McLaughlin wrote:
> > src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java
> > Lines 389-391 (original), 374-376 (patched)
> > <https://reviews.apache.org/r/59699/diff/1-2/?file=1736080#file1736080line391>
> >
> >     We probably don't even need the separate events. We could just have:
> >     
> >         eventSink.post(createDeleteEvent(taskStore, taskIds));
> 
> Kai Huang wrote:
>     So this will change the semantics that: The Delete Event is published after we delete
task from TaskStore?

I don't think it changes the semantics at all? In both cases, the events are published after
the batch delete. All I'm suggesting is we send a single event for the batch rather than one
event per task in the batch.


- David


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59699/#review176670
-----------------------------------------------------------


On June 1, 2017, 9:13 p.m., Kai Huang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59699/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 9:13 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Santhosh Kumar.
> 
> 
> Bugs: AURORA-1929
>     https://issues.apache.org/jira/browse/AURORA-1929
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Improve task history pruning by batch deleting tasks.
> 
> The `'aurora_admin prune_tasks'` endpoint seems to be very slow when the cluster has
a large number of inactive tasks.
> 
> This CR batches all removeTasks operations and execute them all at once to avoid additional
cost of coalescing. The fix will also benefit implicit task history pruning since it has similar
underlying implementation. See https://issues.apache.org/jira/browse/AURORA-1929 for more
information and details.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java 73878210f9028901fda3b08e66c6a63c24260d35

> 
> 
> Diff: https://reviews.apache.org/r/59699/diff/2/
> 
> 
> Testing
> -------
> 
> ./build-support/jenkins/build.sh
> 
> 
> Thanks,
> 
> Kai Huang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message