hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12140) Remove BPOfferService lock contention to get block pool id
Date Fri, 14 Jul 2017 21:45:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088144#comment-16088144
] 

Hudson commented on HDFS-12140:
-------------------------------

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12011 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12011/])
HDFS-12140. Remove BPOfferService lock contention to get block pool id. (kihwal: rev e7d187a1b6a826edd5bd0f708184d48f3674d489)
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java


> Remove BPOfferService lock contention to get block pool id
> ----------------------------------------------------------
>
>                 Key: HDFS-12140
>                 URL: https://issues.apache.org/jira/browse/HDFS-12140
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 2.8.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>             Fix For: 2.9.0, 3.0.0-beta1, 2.8.3
>
>         Attachments: HDFS-12140.branch-2.8.patch, HDFS-12140.trunk.patch
>
>
> The block pool id is protected by a lock in {{BPOfferService}}.  This creates excessive
contention especially for xceivers threads attempting to queue IBRs and heartbeat processing.
 When the latter is delayed due to excessive FSDataset lock contention, it causes pipelines
to collapse.
> Accessing the block pool id should be lockless after registration.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message