Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 9 Feb 2017 18:39:41 +0000 (UTC)
From: "Kihwal Lee (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13038553.1485554899000.41083.1486665581814@Atlassian.JIRA>
In-Reply-To: <JIRA.13038553.1485554899000@Atlassian.JIRA>
References: <JIRA.13038553.1485554899000@Atlassian.JIRA> <JIRA.13038553.1485554899140@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (HADOOP-14033) Reduce fair call queue lock
 contention
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 09 Feb 2017 18:39:50 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-14033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859967#comment-15859967 ] 

Kihwal Lee edited comment on HADOOP-14033 at 2/9/17 6:39 PM:
-------------------------------------------------------------

Is it possible to add a meaningful metrics to show how reader threads (producer) are doing?


was (Author: kihwal):
Is it possible to add a meaningful metrics to show how readers are doing?

> Reduce fair call queue lock contention
> --------------------------------------
>
>                 Key: HADOOP-14033
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14033
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: ipc
>    Affects Versions: 2.7.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: HADOOP-14033.patch
>
>
> Under heavy load the call queue may run dry yet clients experience high latency.
> The fcq requires producers and consumers to sync via a shared lock.  Polling consumers hold the lock while scanning all sub-queues.  Consumers are serialized despite the sub-queues being thread-safe blocking queues.  The effect is to cause other producers/consumers to frequently park.
> The lock is unfair, so producers/consumers attempt to barge in on the lock.  The outnumbered producers tend to remain blocked for an extended time.  As load increases and the queues fill, the barging consumers drain the queues faster than the producers can fill it.
> Server metrics provide an illusion of healthy throughput, response time, and call queue length due to starvation on the ingress.   Often as the load gets worse, the server looks better.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org