hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-13703) Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit
Date Tue, 26 Jun 2018 19:26:00 GMT
Todd Lipcon created HDFS-13703:
----------------------------------

             Summary: Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks
are hit
                 Key: HDFS-13703
                 URL: https://issues.apache.org/jira/browse/HDFS-13703
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: performance
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon


The DFSClient creates a CorruptedBlocks object, which contains a HashMap, on every read call.
In most cases, a read will not hit any corrupted blocks, and this hashmap is not used. It
seems the JIT isn't smart enough to eliminate this allocation. We would be better off avoiding
it and only allocating in the rare case when a corrupt block is hit.

Removing this allocation reduced CPU usage of a TeraValidate job by about 10%.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message