hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12140) Remove BPOfferService lock contention to get block pool id
Date Fri, 14 Jul 2017 19:51:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087922#comment-16087922

Rushabh S Shah commented on HDFS-12140:

[~daryn]: thanks for the patch.
Precommit ran on only branch-2.8 version of patch and not on trunk.
Please re-attach trunk patch again.

I did a code review also.
Overall the patch looks good.
I just have one concern in test case.
You have used mockito.reset few times.
In mockito documentation, it has been mentioned that you shouldn't use reset method. Instead
consider simple and small tests.
But you are testing only small portion of code, so I am +1 to patch.

+1 (non-binding)

> Remove BPOfferService lock contention to get block pool id
> ----------------------------------------------------------
>                 Key: HDFS-12140
>                 URL: https://issues.apache.org/jira/browse/HDFS-12140
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 2.8.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-12140.branch-2.8.patch, HDFS-12140.trunk.patch
> The block pool id is protected by a lock in {{BPOfferService}}.  This creates excessive
contention especially for xceivers threads attempting to queue IBRs and heartbeat processing.
 When the latter is delayed due to excessive FSDataset lock contention, it causes pipelines
to collapse.
> Accessing the block pool id should be lockless after registration.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message