hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Wu (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3672) support for persistent connections to improve pread() performance.
Date Mon, 30 Jun 2008 18:57:45 GMT
support for persistent connections to improve pread() performance.
------------------------------------------------------------------

                 Key: HADOOP-3672
                 URL: https://issues.apache.org/jira/browse/HADOOP-3672
             Project: Hadoop Core
          Issue Type: Improvement
          Components: dfs
    Affects Versions: 0.17.0
         Environment: Linux 2.6.9-55  , Dual Core Opteron 280 2.4Ghz , 4GB memory
            Reporter: George Wu


preads() establish new connections per request. yourkit java profiles show that this connection
overhead is pretty significant on the DataNode. 
I wrote a simple microbenchmark program which does many iterations of pread() from different
offsets of a large file. I hacked DFSClient/DataNode code to re-use the same connection/DataNode
request handler thread. The performance improvement was 7% when the data is served from disk
and 80% when the data is served from the OS page cache.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message