hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "BELUGA BEHR (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-14102) verifyBlockPlacement
Date Tue, 27 Nov 2018 16:07:00 GMT
BELUGA BEHR created HDFS-14102:

             Summary: verifyBlockPlacement
                 Key: HDFS-14102
                 URL: https://issues.apache.org/jira/browse/HDFS-14102
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: BELUGA BEHR

    // 1. Check that all locations are different.
    // 2. Count locations on different racks.
    Set<String> racks = new TreeSet<>();
    for (DatanodeInfo dn : locs)
 Here, the code is counting the number of distinct Network Locations. However, it is using
a TreeSet which has overhead to maintain item order and uses a linked structure internally.
This overhead is unneeded since all that is required here is a count.
{quote}A NavigableSet implementation based on a TreeMap. The elements are ordered using their
natural ordering, or by a Comparator provided at set creation time, depending on which constructor
is used.
 This implementation provides guaranteed log(n) time cost for the basic operations (add, remove
and contains).

 Use Java streams for readability and because it uses a {{HashSet}} under the covers to perform
the distinct action. {{HashSet}} uses an array internally and has constant time performance
for the {{add}} method.


This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message