kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From original-brownbear <...@git.apache.org>
Subject [GitHub] kafka pull request #2775: MINOR: Reduce Fork Count in ITs
Date Fri, 31 Mar 2017 15:48:56 GMT
GitHub user original-brownbear opened a pull request:

    https://github.com/apache/kafka/pull/2775

    MINOR: Reduce Fork Count in ITs

    While looking into instabilities like https://issues.apache.org/jira/browse/KAFKA-4692
I noticed that we are running one JVM fork per core in the ITs.
    
    This seems to me could be contributing to the instability of some ITs a lot. Turning down
the number of forks to half the cores, the IT suits speed up measurably for me (example streams
goes from ~4 minutes to 2.5 minutes).
    Also, case in point this is what 4 forks look like on an i7 (that would get 8 forks in
trunk):
    
    ![4forks](https://cloud.githubusercontent.com/assets/6490959/24557893/a7afa746-1639-11e7-94ff-6327695cf108.png)
    
    Effectively there's still load on all 8 cores at 4 forks (which is expected looking at
the fact that most ITs have 2 or more I/O threads).
    
    => imo it's a good idea to turn this down to half the cores. Let's see how Jenkins
behaves with this in terms of the build times :)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/original-brownbear/kafka fewer-IT-forks

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2775.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2775
    
----
commit 7d88af4c2bc6fe94dced671ec5049c7e2d54e9ec
Author: Armin Braun <me@obrown.io>
Date:   2017-03-31T15:36:17Z

    Minor: Reduce Fork Count in ITs

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message