hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akira AJISAKA (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6945) excessReplicateMap can increase infinitely
Date Wed, 27 Aug 2014 08:32:57 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Akira AJISAKA updated HDFS-6945:
--------------------------------

    Description: 
I'm seeing ExcessBlocks metric increases to more than 300K in some clusters, however, there
are no over-replicated blocks (confirmed by fsck).

After a further research, I noticed when deleting a block, BlockManager does not remove the
block from excessReplicateMap or decrement excessBlocksCount.
Usually the metric is decremented when processing block report, however, if the block has
been deleted, BlockManager does not remove the block from excessReplicateMap or decrement
the metric.
That way the metric and excessReplicateMap can increase infinitely (i.e. memory leak can occur).

  was:I'm seeing ExcessBlocks metric increases to more than 300K in some clusters, however,
there are no over-replicated blocks (confirmed by fsck).

       Priority: Critical  (was: Major)
        Summary: excessReplicateMap can increase infinitely  (was: ExcessBlocks metric may
not be decremented if there are no over replicated blocks)

Updated the summary and the description.

> excessReplicateMap can increase infinitely
> ------------------------------------------
>
>                 Key: HDFS-6945
>                 URL: https://issues.apache.org/jira/browse/HDFS-6945
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Akira AJISAKA
>            Assignee: Akira AJISAKA
>            Priority: Critical
>              Labels: metrics
>         Attachments: HDFS-6945.patch
>
>
> I'm seeing ExcessBlocks metric increases to more than 300K in some clusters, however,
there are no over-replicated blocks (confirmed by fsck).
> After a further research, I noticed when deleting a block, BlockManager does not remove
the block from excessReplicateMap or decrement excessBlocksCount.
> Usually the metric is decremented when processing block report, however, if the block
has been deleted, BlockManager does not remove the block from excessReplicateMap or decrement
the metric.
> That way the metric and excessReplicateMap can increase infinitely (i.e. memory leak
can occur).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message