Return-Path: X-Original-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2A922115E6 for ; Mon, 28 Jul 2014 23:55:41 +0000 (UTC) Received: (qmail 76735 invoked by uid 500); 28 Jul 2014 23:55:40 -0000 Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org Received: (qmail 76685 invoked by uid 500); 28 Jul 2014 23:55:40 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-issues@hadoop.apache.org Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 76672 invoked by uid 99); 28 Jul 2014 23:55:40 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Jul 2014 23:55:40 +0000 Date: Mon, 28 Jul 2014 23:55:40 +0000 (UTC) From: "Jing Zhao (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-10597) Evaluate if we can have RPC client back off when server is under heavy load MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-10597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077154#comment-14077154 ] Jing Zhao commented on HADOOP-10597: ------------------------------------ This can be a very useful feature. For the evaluation, maybe It will be helpful to see how the change will affect the request latency and NN throughput when the workload of NN increases from low to high. For example, one of my questions is that when the NN workload is just relatively high (there are some but not many requests are blocking on the queue), will the average latency and throughput be affected if NN directly lets client back off instead of blocking and timeout? > Evaluate if we can have RPC client back off when server is under heavy load > --------------------------------------------------------------------------- > > Key: HADOOP-10597 > URL: https://issues.apache.org/jira/browse/HADOOP-10597 > Project: Hadoop Common > Issue Type: Sub-task > Reporter: Ming Ma > Assignee: Ming Ma > Attachments: HADOOP-10597-2.patch, HADOOP-10597.patch, RPCClientBackoffDesignAndEvaluation.pdf > > > Currently if an application hits NN too hard, RPC requests be in blocking state, assuming OS connection doesn't run out. Alternatively RPC or NN can throw some well defined exception back to the client based on certain policies when it is under heavy load; client will understand such exception and do exponential back off, as another implementation of RetryInvocationHandler. -- This message was sent by Atlassian JIRA (v6.2#6252)