aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zameer Manji <zma...@apache.org>
Subject Re: Review Request 54011: Add benchmarks for `StateManagerImpl`.
Date Wed, 23 Nov 2016 16:05:26 GMT


> On Nov. 23, 2016, 4:36 a.m., Stephan Erb wrote:
> > Have you considerd writing a single benchmark with a single `run()` method that:
> > 
> > * inserts pending tasks
> > * assigns the pending tasks
> > * deletes the tasks
> > 
> > Even though the benchmark would not be very fine grained, I doubt that we would
lose much precision because of the significant code sharing between all methods. In addtion,
the benchmark would be easy to follow and would not require any setup or cleanup methods.
> > 
> > What do you think?

The problem with this approach is that it becomes dominated by task assignment since you can
only assign one task at a time but insert several thousand tasks or delete several thousand
tasks.

I picked these benchmarks because they clearly show improvements that I can make to insertions
and deletions.

If you insist, I can create this benchmark, but I would make it an adiditional one because
these benchmarks allow me to show improvements to the bulk APIs.


> On Nov. 23, 2016, 4:36 a.m., Stephan Erb wrote:
> > src/jmh/java/org/apache/aurora/benchmark/StateManagerBenchmarks.java, line 82
> > <https://reviews.apache.org/r/54011/diff/1/?file=1568932#file1568932line82>
> >
> >     Just to check my understanding: This will be the in memory task store, correct?

Yes


- Zameer


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54011/#review156727
-----------------------------------------------------------


On Nov. 22, 2016, 6:47 p.m., Zameer Manji wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54011/
> -----------------------------------------------------------
> 
> (Updated Nov. 22, 2016, 6:47 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Stephan Erb.
> 
> 
> Bugs: AURORA-1823
>     https://issues.apache.org/jira/browse/AURORA-1823
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> `StateManagerImpl` is in the middle of every task state transition in the
> scheduler. Performance improvements here could yield scheduling throughput
> improvements across the board. This adds benchmarks for the two bulk APIs,
> inserting pending tasks and deleting tasks. Sample output:
> 
> ````
> Benchmark                                               (numPendingTasks)  (numTasksToDelete)
  Mode  Cnt  Score   Error  Units
> StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A           
    1000  thrpt   10  2.510 ± 0.557  ops/s
> StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A           
   10000  thrpt   10  0.272 ± 0.030  ops/s
> StateManagerBenchmarks.DeleteTasksBenchmark.run                       N/A           
   50000  thrpt   10  0.053 ± 0.011  ops/s
> StateManagerBenchmarks.InsertPendingTasksBenchmark.run               1000           
     N/A  thrpt   10  2.446 ± 0.698  ops/s
> StateManagerBenchmarks.InsertPendingTasksBenchmark.run              10000           
     N/A  thrpt   10  0.246 ± 0.018  ops/s
> StateManagerBenchmarks.InsertPendingTasksBenchmark.run              50000           
     N/A  thrpt   10  0.041 ± 0.006  ops/s
> ````
> 
> 
> Diffs
> -----
> 
>   src/jmh/java/org/apache/aurora/benchmark/StateManagerBenchmarks.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54011/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message