hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravikumar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8892) ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
Date Thu, 13 Aug 2015 11:42:45 GMT
Ravikumar created HDFS-8892:
-------------------------------

             Summary: ShortCircuitCache.CacheCleaner can add Slot.isInvalid() check too
                 Key: HDFS-8892
                 URL: https://issues.apache.org/jira/browse/HDFS-8892
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs-client
    Affects Versions: 2.7.1
            Reporter: Ravikumar
            Priority: Minor


Currently CacheCleaner thread checks only for cache-expiry times. It would be nice if it handles
an invalid-slot too in an extra-pass of evictable map…

for(ShortCircuitReplica replica:evictable.values()) {
     if(!scr.getSlot().isValid()) {
        purge(replica);
     }
}
//Existing code...
int numDemoted = demoteOldEvictableMmaped(curMs);
int numPurged = 0;
Long evictionTimeNs = Long.valueOf(0);
….
…..

Apps like HBase can tweak the expiry/staleness/cache-size params in DFS-Client, so that ShortCircuitReplica
will never be closed except when Slot is declared invalid. 

I assume slot-invalidation will happen during block-invalidation/deletes {Primarily triggered
by compaction/shard-takeover etc..}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message