hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8794) Improve CorruptReplicasMap#corruptReplicasMap
Date Fri, 17 Jul 2015 22:46:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632063#comment-14632063

Yi Liu commented on HDFS-8794:

Thanks [~arpitagarwal] for review!

Actually I have added {{@VisibleForTesting}} annotation for {{getCorruptReplicaBlockIds}},
I can certainly rename it as you suggestion in next patch.

It looks like {{getCorruptReplicaBlockIds}} won't work in general since it assumes the map
is sorted
You are right,  I have fixed this too in the current patch, so no need additional fix, look
at following code, it uses a {{TreeMap}} wrapper in {{getCorruptReplicaBlockIds}}.  The Jenkins
result also shows the fix correct.
+    Iterator<Block> blockIt = 
+        new TreeMap<>(corruptReplicasMap).keySet().iterator();

> Improve CorruptReplicasMap#corruptReplicasMap
> ---------------------------------------------
>                 Key: HDFS-8794
>                 URL: https://issues.apache.org/jira/browse/HDFS-8794
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>         Attachments: HDFS-8794.001.patch
> Currently we use {{TreeMap}} for {{corruptReplicasMap}}, actually the only need sorted
place is {{getCorruptReplicaBlockIds}} which is used by test.
> So we can use {{HashMap}}.
> From memory and performance view, {{HashMap}} is better than {{TreeMap}}, a simliar optimization
HDFS-7433. Of course we need to make few change to {{getCorruptReplicaBlockIds}}.

This message was sent by Atlassian JIRA

View raw message