camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Müller (JIRA) <j...@apache.org>
Subject [jira] [Comment Edited] (CAMEL-5918) Add performance tests
Date Sat, 05 Jan 2013 15:38:12 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541362#comment-13541362
] 

Christian Müller edited comment on CAMEL-5918 at 1/5/13 3:36 PM:
-----------------------------------------------------------------

I could measure the following performance in my environment:
{noformat}
cmueller$ /usr/sbin/system_profiler SPHardwareDataType
Hardware:
    Hardware Overview:
      Model Name: MacBook Pro
      Model Identifier: MacBookPro5,2
      Processor Name: Intel Core 2 Duo
      Processor Speed: 3.06 GHz
      Number of Processors: 1
      Total Number of Cores: 2
      L2 Cache: 6 MB
      Memory: 8 GB
      Bus Speed: 1.07 GHz

Java version: 1.6.0_37, vendor: Apple Inc.
OS name: "mac os x", version: "10.8.2", arch: "x86_64", family: "mac"
{noformat}

||Test||Camel 2.8.7-SNAPSHOT||Camel 2.9.6-SNAPSHOT||Camel 2.10.4-SNAPSHOT||Camel 2.11-SNAPSHOT||
|ContentBasedRoutingPerformanceTest.testChoiceSimple|2516ms|2699ms|2608ms (659ms)|2266ms (419ms)|
|ContentBasedRoutingPerformanceTest.testChoiceExpression|2426ms|2273ms|2352ms (633ms)|2155ms
(379ms)|
|ContentBasedRoutingPerformanceTest.testFilterSimple|2977ms|2368ms|2353ms (565ms)|2116ms (359ms)|
|ContentBasedRoutingPerformanceTest.testFilterExpression|2416ms|2922ms|2365ms (554ms)|2161ms
(371ms)|
|HeaderBasedRoutingPerformanceTest.testChoiceSimple|2434ms|2513ms|2371ms (593ms)|2221ms (390ms)|
|HeaderBasedRoutingPerformanceTest.testChoiceExpression|2484ms|2416ms|2433ms (578ms)|2214ms
(420ms)|
|HeaderBasedRoutingPerformanceTest.testFilterSimple|2456ms|2463ms|2411ms (594ms)|2184ms (381ms)|
|HeaderBasedRoutingPerformanceTest.testFilterExpression|2453ms|2496ms|2408ms (589ms)|2225ms
(379ms)|
|SplitterPerformanceTest.testTokenize|50196ms|{color:green}18912ms{color}|18511ms (876ms)|17488ms
(897ms)|
|XPathBasedRoutingPerformanceTest.testChoice|6207ms|6194ms|6394ms (4391ms)|5904ms (4266ms)|
|XPathBasedRoutingPerformanceTest.testFilter|6206ms|6191ms|6409ms (4366ms)|5925ms (4133ms)|
|XQueryBasedRoutingPerformanceTest.testChoice|4979ms|4943ms|4868ms (3247ms)|4668ms (2870ms)|
|XQueryBasedRoutingPerformanceTest.testFilter|4819ms|4887ms|4872ms (3041ms)|4693ms (2852ms)|
|XsltPerformanceTest.testXslt|3337ms|3678ms|3407ms (2795ms)|3193ms (2677ms)|
|ProxyEsbPerformanceIntegrationTest.testProxy||||32773ms|
|HttpHbrEsbPerformanceIntegrationTest.testHttpHbr||||24856ms|
|CbrEsbPerformanceIntegrationTest.testCbr||||38129ms|
|SoapHbrEsbPerformanceIntegrationTest.testSoapHbr||||39254ms|
|SecureProxyEsbPerformanceIntegrationTest.testSecureProxy||||148452ms|
|XsltEsbPerformanceIntegrationTest.testXSLTProxy||||44257ms|

Starting with Camel 2.10.x, I use {{mock.setRetainFirst(0)}} and {{mock.setRetainLast(0)}}
to speed up the tests. These performance numbers are shown in brackets.
                
      was (Author: muellerc):
    I could measure the following performance in my environment:
{noformat}
cmueller$ /usr/sbin/system_profiler SPHardwareDataType
Hardware:
    Hardware Overview:
      Model Name: MacBook Pro
      Model Identifier: MacBookPro5,2
      Processor Name: Intel Core 2 Duo
      Processor Speed: 3.06 GHz
      Number of Processors: 1
      Total Number of Cores: 2
      L2 Cache: 6 MB
      Memory: 8 GB
      Bus Speed: 1.07 GHz

Java version: 1.6.0_37, vendor: Apple Inc.
OS name: "mac os x", version: "10.8.2", arch: "x86_64", family: "mac"
{noformat}

||Test||Camel 2.8.7-SNAPSHOT||Camel 2.9.6-SNAPSHOT||Camel 2.10.4-SNAPSHOT||Camel 2.11-SNAPSHOT||
|ContentBasedRoutingPerformanceTest.testChoiceSimple|2516ms|2699ms|2608ms (659ms)|2266ms (419ms)|
|ContentBasedRoutingPerformanceTest.testChoiceExpression|2426ms|2273ms|2352ms (633ms)|2155ms
(379ms)|
|ContentBasedRoutingPerformanceTest.testFilterSimple|2977ms|2368ms|2353ms (565ms)|2116ms (359ms)|
|ContentBasedRoutingPerformanceTest.testFilterExpression|2416ms|2922ms|2365ms (554ms)|2161ms
(371ms)|
|HeaderBasedRoutingPerformanceTest.testChoiceSimple|2434ms|2513ms|2371ms (593ms)|2221ms (390ms)|
|HeaderBasedRoutingPerformanceTest.testChoiceExpression|2484ms|2416ms|2433ms (578ms)|2214ms
(420ms)|
|HeaderBasedRoutingPerformanceTest.testFilterSimple|2456ms|2463ms|2411ms (594ms)|2184ms (381ms)|
|HeaderBasedRoutingPerformanceTest.testFilterExpression|2453ms|2496ms|2408ms (589ms)|2225ms
(379ms)|
|SplitterPerformanceTest.testTokenize|{color:red}50196ms{color}|18912ms|18511ms (876ms)|17488ms
(897ms)|
|XPathBasedRoutingPerformanceTest.testChoice|6207ms|6194ms|6394ms (4391ms)|5904ms (4266ms)|
|XPathBasedRoutingPerformanceTest.testFilter|6206ms|6191ms|6409ms (4366ms)|5925ms (4133ms)|
|XQueryBasedRoutingPerformanceTest.testChoice|4979ms|4943ms|4868ms (3247ms)|4668ms (2870ms)|
|XQueryBasedRoutingPerformanceTest.testFilter|4819ms|4887ms|4872ms (3041ms)|4693ms (2852ms)|
|XsltPerformanceTest.testXslt|3337ms|3678ms|3407ms (2795ms)|3193ms (2677ms)|

Starting with Camel 2.10.x, I use {{mock.setRetainFirst(0)}} and {{mock.setRetainLast(0)}}
to speed up the tests. These performance numbers are shown in brackets.
                  
> Add performance tests
> ---------------------
>
>                 Key: CAMEL-5918
>                 URL: https://issues.apache.org/jira/browse/CAMEL-5918
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-test
>    Affects Versions: 2.9.5, 2.10.3
>            Reporter: Christian Müller
>            Assignee: Christian Müller
>             Fix For: 2.8.7, 2.9.6, 2.10.4, 2.11.0
>
>
> By working on the "ESB Performance Testing - Round 7" (http://www.esbperformance.org/)
to provide a good/optimal ServiceMix & Camel test set up I figured out, the performance
for most of the test cases are worse by upgrading from ServiceMix 4.4.2 (which use Camel 2.8.5)
to ServiceMix 4.5.0-SNAPSHOT (which use Camel 2.10.3).
> The reason for this does not have to be a worse performance in Camel, but it could. In
these performance tests, I use ServiceMix/Karaf, Jetty, CXF, ... and Camel of course.
> To make sure the performance is not getting worse in a new Camel release, I will add
a few different performance test in the next days/weeks for the most relevant use cases and
the tests made by esbperformance.org. These test should not be executed by default on the
CI server. But we should run the tests regularly from time to time and before we cut a new
release. We also can use these tests to profile Camel and check we can improve the performance,
independently how good the performance already is.
> I'm wondering what is the best place for this module. I suggest camel/tests/camel-performance.
I don't know what the intension was for the existing camel/tests/camel-itest-performance module?
It doesn't contain performance tests...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message