hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chengxiang Li (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-5276) FileSystem.Statistics got performance issue on multi-thread read/write.
Date Sun, 29 Sep 2013 03:12:23 GMT
Chengxiang Li created HDFS-5276:
-----------------------------------

             Summary: FileSystem.Statistics got performance issue on multi-thread read/write.
                 Key: HDFS-5276
                 URL: https://issues.apache.org/jira/browse/HDFS-5276
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.0.4-alpha
            Reporter: Chengxiang Li


FileSystem.Statistics is a singleton variable for each FS scheme, each read/write on HDFS
would lead to a AutomicLong.getAndAdd(). AutomicLong does not perform well in multi-threads(let's
say more than 30 threads). so it may cause  serious performance issue. during our test profile,
32 threads read data from HDFS, about 70% cpu time is spent on FileSystem.Statistics.incrementBytesRead().



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message