hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Zeller (JIRA)" <j...@apache.org>
Subject [jira] Created: (HDFS-492) Expose corrupt replica/block information
Date Wed, 15 Jul 2009 18:32:14 GMT
Expose corrupt replica/block information

                 Key: HDFS-492
                 URL: https://issues.apache.org/jira/browse/HDFS-492
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: data-node, name-node
    Affects Versions: 0.21.0
            Reporter: Bill Zeller
            Priority: Minor

This adds two additional functions to FSNamesystem to provide more information about corrupt
replicas. It also adds two servlets to the namenode that provide information (in JSON) about
all blocks with corrupt replicas as well as information about a specific block. It also changes
the file browsing servlet by adding a link from block ids to the above mentioned block information

These JSON pages are designed to be used by client side tools which wish to analyze corrupt
block/replicas. The only change to an existing (non-servlet) class is described below).  

Currently, CorruptReplicasMap stores a map of corrupt replica information and allows insertion
and deletion. It also gives information about the corrupt replicas for a specific block. It
does not allow iteration over all corrupt blocks. Two additional functions will be added to
FSNamesystem (which will call BlockManager which will call CorruptReplicasMap). The first
will return the size of the corrupt replicas map, which represents the number of blocks that
have corrupt replicas (but less than the number of corrupt replicas if a block has multiple
corrupt replicas). The second will allow "paging" through a list of block ids that contain
corrupt replicas:

{{  public synchronized List<Long> getCorruptReplicaBlockIds(int n, Long startingBlockId)

{{n}} is the number of block ids to return and {{startingBlockId}} is the block id offset.
To prevent a large number of items being returned at one time, n is constrained to 0 <=
{{n}} <= 100. If {{startingBlockId}} is null, up to {{n}} items are returned starting at
the beginning of the list. Ordering is enforced through the internal use of TreeMap in CorruptReplicasMap.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message