flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang, Yanping" <yanping.w...@intel.com>
Subject RE: [jira] [Created] (FLINK-2343) Change default garbage collector in streaming environments
Date Sun, 12 Jul 2015 14:38:05 GMT
Hi, Stephan

I'd like to promote G1 as default GC for Flink with following reasons
(1) CMS is no longer the development focus from Oracle Java and OpenJDK, all the GC development
is focusing on G1
(2) Based on jdk9 plan, jdk9's default collector for jdk9 release will be G1
(3) jdk8 update 45 is now the default jdk to be downloaded  from http://www.oracle.com/technetwork/java/javase/downloads/index.html
In jdk8 update 45, G1 is in full production with many optimizations fit for Spark, etc running
100GB plus heap.
(4) In addition, upcoming release of jdk8 update 60 will contain following optimizations on:

•	Refactor Remembered Set for Object Array
•	Early reclaim dead humongous objects
•	Hot card cache lock contention reduction
•	ext root scan optimization (scanning the roots: globals, registers, thread stacks and
VM data structures)
•	class unloading during the marking cycle 
•	object copy optimization
•	G1 default value adjustment for some global flags
•	Dynamic IHOP
•	Fix PLAB sizing heuristic

I used jdk8 update 60 (downloaded from here: https://jdk8.java.net/download.html  )  measured
2X performance improvement over jdk7 update 79 on one Spark graph computing workload running
4 node clusters of Intel Ivybridge servers with 160GB java heap.

I was the one of several JVM runtime engineers who helped putting those optimizations for
Spark in jdk8 update 40 and above. Let me know if you have any questions regarding G1 status
and how to best run G1.

-----Original Message-----
From: Stephan Ewen (JIRA) [mailto:jira@apache.org] 
Sent: Friday, July 10, 2015 7:36 AM
To: dev@flink.apache.org
Subject: [jira] [Created] (FLINK-2343) Change default garbage collector in streaming environments

Stephan Ewen created FLINK-2343:

             Summary: Change default garbage collector in streaming environments
                 Key: FLINK-2343
                 URL: https://issues.apache.org/jira/browse/FLINK-2343
             Project: Flink
          Issue Type: Improvement
          Components: Start-Stop Scripts
    Affects Versions: 0.10
            Reporter: Stephan Ewen
            Assignee: Stephan Ewen
             Fix For: 0.10

When starting Flink, we don't pass any particular GC related JVM flags to the system. That
means, it uses the default garbage collectors, which are the bulk parallel GCs for both old
gen and new gen.

For streaming applications, this results in vastly fluctuating latencies. Latencies are much
more constant with either the {{CMS}} or {{G1}} GC.

I propose to make the CMS the default GC for streaming setups.

G1 may become the GC of choice in the future, but fro various articles I found, it is still
somewhat in "beta" status (see for example here: http://jaxenter.com/kirk-pepperdine-on-the-g1-for-java-9-118190.html

This message was sent by Atlassian JIRA

View raw message