aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zameer Manji <zma...@apache.org>
Subject Re: Review Request 43457: Increase throughput of DbTaskStore
Date Thu, 11 Feb 2016 02:33:23 GMT


> On Feb. 10, 2016, 4:23 p.m., John Sirois wrote:
> > > and removes the population of an object via a constructor which is slower than
populating an object via setters.
> > 
> > For clarification, is this optimization the much smaller of the 2?  Naively - I'd
hope construction would beat setters, but - if not - be _very close_.  If this latter bit
was significant, my faith in the jvm is duly shaken.
> 
> Zameer Manji wrote:
>     Switching to Pair and populating via constructor:
>     ````
>     Benchmark                                      (numTasks)   Mode  Cnt   Score   
Error  Units
>     TaskStoreBenchmarks.DBFetchTasksBenchmark.run       10000  thrpt    5  53.040 ±
17.152  ops/s
>     TaskStoreBenchmarks.DBFetchTasksBenchmark.run       50000  thrpt    5   3.939 ±
 3.357  ops/s
>     TaskStoreBenchmarks.DBFetchTasksBenchmark.run      100000  thrpt    5   0.079 ±
 0.020  ops/s
>     ````
>     
>     I suspect it is because if you populate via constructor you cannot use `<id>`.
> 
> John Sirois wrote:
>     Fwict only the 100000 case has a statistically significant win (the error bars in
the 2nd run are quite massive) - is that worth it?  Does Aurora actually fetch 100K tasks
for any operation in practice at Twitter?

Yes this change is worth it. At Twitter Aurora actually fetches at least 100K tasks for some
operations. This is a side effect of some unscoped `fetchTask` queries we currently have.


- Zameer


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43457/#review118772
-----------------------------------------------------------


On Feb. 10, 2016, 4:35 p.m., Zameer Manji wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43457/
> -----------------------------------------------------------
> 
> (Updated Feb. 10, 2016, 4:35 p.m.)
> 
> 
> Review request for Aurora, John Sirois and Maxim Khutornenko.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Profiling master indicated that the bottleneck was MyBatis populating ResultSets and
populating the resulting objects. This patch removes subselects, which reduces the number
of ResultSets and removes the population of an object via a constructor which is slower than
populating an object via setters.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbAssginedPort.java PRE-CREATION

>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbAssignedTask.java 93722395ed9fcd22dcb12e34e648e6e410952d43

>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbScheduledTask.java 502a1fa6fc141df498f0f09af292ce24e269731d

>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.xml b1394cf44b7ddafcbc47bb1968306d0b33293380

>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml ea469cce31544221c34ae05a1c65f71271985655

> 
> Diff: https://reviews.apache.org/r/43457/diff/
> 
> 
> Testing
> -------
> 
> Master:
> Benchmark                                      (numTasks)   Mode  Cnt   Score    Error
 Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       10000  thrpt    5  44.052 ± 14.689
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       50000  thrpt    5   0.179 ±  0.052
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run      100000  thrpt    5   0.087 ±  0.022
 ops/s
> 
> This Patch:
> Benchmark                                      (numTasks)   Mode  Cnt   Score   Error
 Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       10000  thrpt    5  51.531 ± 7.236
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       50000  thrpt    5   7.370 ± 1.320
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run      100000  thrpt    5   2.143 ± 1.234
 ops/s
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message