Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <649779795.1207001544531.JavaMail.jira@brutus>
Date: Mon, 31 Mar 2008 15:12:24 -0700 (PDT)
From: "Sameer Paranjpye (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3109) RPC should accepted connections
 even when rpc queue is full (ie undo part of HADOOP-2910)
In-Reply-To: <102648156.1206644606780.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583901#action_12583901 ] 

Sameer Paranjpye commented on HADOOP-3109:
------------------------------------------

Managing the backlog in a portable way is not easy. Many Linux versions will truncate the backlog to 128 (silently) if it is set higher, for example.


> RPC should accepted connections even when rpc queue is full (ie undo part of HADOOP-2910)
> -----------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3109
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3109
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Sanjay Radia
>            Assignee: Hairong Kuang
>            Priority: Blocker
>             Fix For: 0.17.0
>
>
> HADOOP-2910 changed HDFS to stop accepting new connections when the rpc queue is full. It should continue to accept connections and let the OS  deal with limiting connections.
> HADOOP-2910's decision to not read from open sockets when queue is full is exactly right -  backup on the
> client sockets and they will just wait( especially with HADOOP-2188 that removes client timeouts).
> However we should continue to  accept connections:
> The OS refuses new connections after a large number of connections are open (this is configurable parameter). With this patch, we have  new lower limit for # of open connections when the RPC queue is full.
> The problem is that when there is a surge of requests, we would stop
> accepting connection and clients will get a connection failed (a change from old behavior).
> Instead if you continue to accept connections it is likely that the surge will be over shortly and
> clients will get served. Of course if the surge lasts a long time the OS will stop accepting connections
> and clients will fail and there not much one can do (except raise the os limit).
> I propose that we continue accepting connections, but not read from
> connections when the RPC queue is full. (ie undo part of 2910 work back to the old behavior).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.