hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-13594) the lock of ShortCircuitCache is hold while close the ShortCircuitReplica
Date Fri, 18 May 2018 10:44:00 GMT
Gang Xie created HDFS-13594:
-------------------------------

             Summary: the lock of ShortCircuitCache is hold while close the ShortCircuitReplica
 
                 Key: HDFS-13594
                 URL: https://issues.apache.org/jira/browse/HDFS-13594
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs-client
    Affects Versions: 3.0.2
            Reporter: Gang Xie
         Attachments: no_hdfs.svg

When we are profiling SC read, we find that ShortCircuitCache's lock is a hot spot. After
look into the code, we find that when close BlockReaderLocal, it tries to  trimEvictionMaps,
and several ShortCircuitReplicas are closed while the lock being hold.  This slows down
the close of the BlockReaderLocal, and the worse is that it blocks the other allocating of
the new ShortCircuitReplicas. 

An idea to avoid this is to close the replica in an async way. I will do a prototype and get
the performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message