hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4562) Many excess replicas getting created
Date Thu, 07 Mar 2013 08:16:19 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13595664#comment-13595664

Amareshwari Sriramadasu commented on HDFS-4562:

bq. Can you also check, whether the DNs registration are getting expired due to some reason
and they are rejoing again in the cluster? [should see the logs like 'Adding a new node']

No. We are not seeing this. 

bq. we can check any node is very low responsive and NN is replicating that blocks before
he reports.
we are not seeing there is any particular datanode that is responding slow. 

Looking at my first comment again, the block was allocated at 00:02:21,787. 
First addStoredBlock at 00:02:21,794, second at 00:02:21,794. 
Replication request at 00:02:21,803. And third report at the same time - 00:02:21,803.

This says the replication monitor is too aggressive. Is there any other way to control this?

> Many excess replicas getting created
> ------------------------------------
>                 Key: HDFS-4562
>                 URL: https://issues.apache.org/jira/browse/HDFS-4562
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 1.1.1
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 1.2.0
> We are seeing too many excess replicas getting created in our cluster. The number excess
replicas in day coming out to be more than 1 lakh.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message