aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zameer Manji <zma...@apache.org>
Subject Re: Review Request 43457: Increase throughput of DbTaskStore
Date Thu, 11 Feb 2016 19:59:02 GMT


> On Feb. 10, 2016, 4:33 p.m., Maxim Khutornenko wrote:
> > src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml, line 83
> > <https://reviews.apache.org/r/43457/diff/1/?file=1240027#file1240027line83>
> >
> >     You may improve perf even further if you include `id` element here:
> >     
> >     "Very Important: id elements play a very important role in Nested Result mapping.
You should always specify one or more properties that can be used to uniquely identify the
results. The truth is that MyBatis will still work if you leave it out, but at a severe performance
cost. Choose as few properties as possible that can uniquely identify the result. The primary
key is an obvious choice (even if composite)."
> >     
> >     http://www.mybatis.org/mybatis-3/sqlmap-xml.html#Result_Maps
> 
> John Sirois wrote:
>     If Maxim's suggestion works, I'd rather have that fix on the books if just to help
suggest <collection/> mappings should be scrutinized with care.  We're using MyBatis
- I think - to be able to go lower-level than - say Hibernate - but clearly the same pitfalls
are exist!
> 
> John Sirois wrote:
>     Erm, aha - LGPLv2... which I'm very happy with, but Apache is not :/

There isn't a performance improvement here probably because of the `<id>` element in
the `portMap` resultMap.


> On Feb. 10, 2016, 4:33 p.m., Maxim Khutornenko wrote:
> > src/main/java/org/apache/aurora/scheduler/storage/db/views/DbScheduledTask.java,
line 42
> > <https://reviews.apache.org/r/43457/diff/1/?file=1240025#file1240025line42>
> >
> >     Will `ORDER BY te_timestamp` accomplish the same without explicit sorting in
the mapper? If order invariant is perserved AND there is no perf impact query change may be
preferrable (less code).

The `ORDER BY` does not accomplish the same thing for some reason. I will instead reaplce
the sortedCopy with an inplace copy.


- Zameer


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43457/#review118774
-----------------------------------------------------------


On Feb. 10, 2016, 4:35 p.m., Zameer Manji wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43457/
> -----------------------------------------------------------
> 
> (Updated Feb. 10, 2016, 4:35 p.m.)
> 
> 
> Review request for Aurora, John Sirois and Maxim Khutornenko.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Profiling master indicated that the bottleneck was MyBatis populating ResultSets and
populating the resulting objects. This patch removes subselects, which reduces the number
of ResultSets and removes the population of an object via a constructor which is slower than
populating an object via setters.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbAssginedPort.java PRE-CREATION

>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbAssignedTask.java 93722395ed9fcd22dcb12e34e648e6e410952d43

>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbScheduledTask.java 502a1fa6fc141df498f0f09af292ce24e269731d

>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.xml b1394cf44b7ddafcbc47bb1968306d0b33293380

>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml ea469cce31544221c34ae05a1c65f71271985655

> 
> Diff: https://reviews.apache.org/r/43457/diff/
> 
> 
> Testing
> -------
> 
> Master:
> Benchmark                                      (numTasks)   Mode  Cnt   Score    Error
 Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       10000  thrpt    5  44.052 ± 14.689
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       50000  thrpt    5   0.179 ±  0.052
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run      100000  thrpt    5   0.087 ±  0.022
 ops/s
> 
> This Patch:
> Benchmark                                      (numTasks)   Mode  Cnt   Score   Error
 Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       10000  thrpt    5  51.531 ± 7.236
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run       50000  thrpt    5   7.370 ± 1.320
 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run      100000  thrpt    5   2.143 ± 1.234
 ops/s
> 
> 
> Thanks,
> 
> Zameer Manji
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message