hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yingqi Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8134) Using OpenJDK on HDFS
Date Thu, 30 Apr 2015 23:49:06 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yingqi Lu updated HDFS-8134:
----------------------------
    Attachment: pic2.png
                pic1.png

Hi All,

Here are the most recent data on OpenJDK with HDFS. 

Purpose of the study: 
The goal is to show OpenJDK is enterprise ready from the performance point of view.  

Configuration:
•	Intel Xeon® E5-2699 V3 (2 X 18Core 2.3Ghz) CPUs
•	BIOS  Version: SE5C610.86B.01.01.0008.021120151325 (release date 02/11/2015)
•	All BIOS settings are kept default (HT enabled, Turbo enabled, Power features enabled)
•	Memory: 16 X 16GB DDR4 2133MHz, 2 Dimms per channel
•	Storage: OS is installed on a 120GB SSD. HDFS and tmp directory is located on 1 PCIeSSD
drive (1 X Intel® SSD DC P3700 Series, 1/2 Height PCIe 3.0, 20nm, MLC)
•	OS: CentOS 7 kernel version 3.10.0-123.el7.x86_64
•	Hadoop: 3.0.0-SNAPSHOT (commit 867d5d2675b8fb73c40fac1e581b02b005459d95, dated 04/02/2015),
single node cluster
•	Java: Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode) vs. OpenJDK 64-Bit
Server VM (build 25.40-b25, mixed mode). Two GC methods are checked – G1GC and ParallelOldGC.
•	Workload: dfsioe inside HiBench suite. Data size is 128M. Read and write operations are
included in the performance data study.

Data summary:
1.	With 128M file size and 1000 files, both read and write operations from dfsioe show similar
performance (throughput) between Hotspot Java and OpenJDK (5% performance difference is within
workload run-run variance range). 
2.	We also tested 2 GC methods – G1Gc and ParallelOldGC. They both show similar performance
as well for this specific workload.
3.	The single system cluster runs 95%+ CPU utilization for both read and write operations.

Performance charts are attached here. Please let me know if you have any questions and comments.

Thanks,
Yingqi Lu


> Using OpenJDK on HDFS
> ---------------------
>
>                 Key: HDFS-8134
>                 URL: https://issues.apache.org/jira/browse/HDFS-8134
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: benchmarks, performance
>         Environment: CentOS7, OpenJDK8 update 40, Oracle JDK8 update 40
>            Reporter: Yingqi Lu
>            Assignee: Yingqi Lu
>         Attachments: pic1.png, pic2.png
>
>
> Dear All,
> We would like to start the effort of certifying OpenJDK with HDFS. The effort includes
compiling HDFS source code with OpenJDK and reporting issues if there is any, and completing
performance study and comparing all the results with Oracle JDK. The workload we will start
with is DFSIOe which is part of the HiBench suite. We can surely add more workloads such as
Teragen and etc. into our testing environment if there is any interest from this community.

> This is our first time to work on this community. Please do let us know your feedback
and comments. If you all like the idea and this is the right place to start the effort, we
will be sending out the data soon!
> Thanks,
> Yingqi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message