Return-Path: X-Original-To: apmail-hc-httpclient-users-archive@www.apache.org Delivered-To: apmail-hc-httpclient-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C9D9D18511 for ; Sat, 22 Aug 2015 08:08:25 +0000 (UTC) Received: (qmail 78682 invoked by uid 500); 22 Aug 2015 08:08:25 -0000 Delivered-To: apmail-hc-httpclient-users-archive@hc.apache.org Received: (qmail 78634 invoked by uid 500); 22 Aug 2015 08:08:25 -0000 Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "HttpClient User Discussion" Delivered-To: mailing list httpclient-users@hc.apache.org Received: (qmail 78620 invoked by uid 99); 22 Aug 2015 08:08:25 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Aug 2015 08:08:25 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id B07361827CC for ; Sat, 22 Aug 2015 08:08:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.68 X-Spam-Level: X-Spam-Status: No, score=0.68 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id M3JSGnW7hTza for ; Sat, 22 Aug 2015 08:08:07 +0000 (UTC) Received: from mail-wi0-f173.google.com (mail-wi0-f173.google.com [209.85.212.173]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 50D2D506F3 for ; Sat, 22 Aug 2015 07:51:03 +0000 (UTC) Received: by wicja10 with SMTP id ja10so31466176wic.1 for ; Sat, 22 Aug 2015 00:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:mime-version:subject :message-id:date:references:in-reply-to:to; bh=hThNgjL5sX6NM+AD4ZA1lPcCgGOlrjAwcxA6EhdnMW4=; b=zv9Uiw8zCHNGPooIQKtnqagnLTxedXZnhvasnDHexfnusa17zrrmOO5Q11fuwExrEc uFWBWwWxxHozO6HYuJTw7o7f7doACOHLWpR334tMkkJuhqINJn2GBvW86cuTM8BIz+QI G2IYoe1wKZivHFCY3+9DbL2soxGdfz2KStM7jb2iSgksR5keqFPxXICj+kTm3PEbcBF4 iDZvgqKMnsuGs1YwIHlOIthr2pu/xqhaf/IPxBD0EeY9vRSGVe/43imrnK+8YhfiPFxc Vc7BmClF3Nceyr6igmaEr1ex8Axtfu1scOVOaNYZIJdEPq1YxeDHrlEjtRV1iwVslVBD eISA== X-Received: by 10.180.188.131 with SMTP id ga3mr12921279wic.69.1440229862569; Sat, 22 Aug 2015 00:51:02 -0700 (PDT) Received: from [10.0.1.5] (10.79-161-19.customer.lyse.net. [79.161.19.10]) by smtp.gmail.com with ESMTPSA id fa8sm6210320wib.14.2015.08.22.00.51.01 for (version=TLSv1/SSLv3 cipher=OTHER); Sat, 22 Aug 2015 00:51:01 -0700 (PDT) From: =?utf-8?Q?Stefan_Magnus_Landr=C3=B8?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Subject: Re: intermittent org.apache.http.ConnectionClosedException: Message-Id: <1CED26C3-9E80-4B95-96AD-215E1ED8FD8E@gmail.com> Date: Sat, 22 Aug 2015 09:51:00 +0200 References: <1440080148.16880.0.camel@apache.org> <1440080712.16880.2.camel@apache.org> <1440081130.16880.4.camel@apache.org> <1440083401.23312.1.camel@apache.org> <0050FE97-9378-494F-9C3F-ECAD6EC6CD40@gmail.com> <276EB239-030D-4815-A903-F44EC8C704A6@gmail.com> In-Reply-To: To: HttpClient User Discussion X-Mailer: iPhone Mail (12H321) Have a look at the sources to see what http client does when closed.=20 Check with jvm to see what happens when sockets are closed Sendt fra min iPhone > Den 22. aug. 2015 kl. 08.14 skrev Shushant Arora : >=20 > What is the state of connections when all sent requests are completed and I= > don't call close on httpclient. >=20 > 1.When requests are being sent and I observe on shell using $ ss -ta - i > can see connections in Established-sync send state. But when no requests > are sent - nothing is displayed - connections count is 0 and i never see > any connection transit to TIME_WAIT or close state how connections count > reduced to 0 without transitioning to close state ? >=20 > But when I call client.close after requests are complted - connections do= > transit to close state and wait there till fin_time (~1 min ) and get > terminates. >=20 > On Fri, Aug 21, 2015 at 10:25 AM, Stefan Magnus Landr=C3=B8 < > stefan.landro@gmail.com> wrote: >=20 >>=20 >>=20 >> Sendt fra min iPhone >>=20 >>>> Den 21. aug. 2015 kl. 00.39 skrev Shushant Arora < >>> shushantarora09@gmail.com>: >>>=20 >>> Thanks ! >>>=20 >>> shall the value of tcp_fin_timeout be reduced(from default 60 to 30) in >>> heavy concurrent requests? >>=20 >> Heavy? What do you mean? >>=20 >>> Does it have any side affect? >>=20 >> Seriously? >>=20 >>>=20 >>> On Fri, Aug 21, 2015 at 2:12 AM, Stefan Magnus Landr=C3=B8 < >>> stefan.landro@gmail.com> wrote: >>>=20 >>>>=20 >>>>=20 >>>> Sendt fra min iPhone >>>>=20 >>>>>> Den 20. aug. 2015 kl. 21.06 skrev Shushant Arora < >>>>> shushantarora09@gmail.com>: >>>>>=20 >>>>> A doubt regarding async client - does it sends the request on same >>>>> connection before waiting for repose of previous request on same >>>> connection >>>>=20 >>>> No request multiplexing in http 1.1 >>>>=20 >>>>> ? Or it will reuse the connection only when first request's response >> has >>>>> arrived or request timeouts ? >>>>>=20 >>>>> Since I doubt connection close is not coming because of server rather= >>>>> client machine tcp socket connections limit - Same http client with >> same >>>>> pool size to same same server is working fine but giving connection >> close >>>>> exception on another machine. >>>>>=20 >>>>>=20 >>>>> When I saw the value of sysctl net.ipv4.ip_local_port_range >>>>> On machine where exceptions are coming its >>>>> net.ipv4.ip_local_port_range =3D 32768 61000 >>>>> but on another machibe its net.ipv4.ip_local_port_range =3D 1024 >> 65500 >>>>>=20 >>>>>> On Thu, Aug 20, 2015 at 8:40 PM, Oleg Kalnichevski = >>>> wrote: >>>>>>=20 >>>>>>> On Thu, 2015-08-20 at 20:31 +0530, Shushant Arora wrote: >>>>>>> but its failing my job .Is there any eviction policy I should be >>>> handling >>>>>>> or its automatically taken care of evicting the closed connections. >>>>>>=20 >>>>>> You have not been listening, have you? The server closes the >> connection >>>>>> on its end _prematurely_. >>>>>>=20 >>>>>> Oleg >>>>>>=20 >>>>>>>> On Thu, Aug 20, 2015 at 8:02 PM, Oleg Kalnichevski < >> olegk@apache.org> >>>>>>> wrote: >>>>>>>=20 >>>>>>>>> On Thu, 2015-08-20 at 20:00 +0530, Shushant Arora wrote: >>>>>>>>> Unplugging is not possible. >>>>>>>>>=20 >>>>>>>>> How to capture the exception in code and detect if a connection is= >>>>>>>> closed - >>>>>>>>> don't use it in further requests and create new one in pool (set t= o >>>>>> max >>>>>>>>> connnection pool)? >>>>>>>>=20 >>>>>>>> HttpClient manages persistent connections internally and >> automatically >>>>>>>> discards connections that cannot be re-used. >>>>>>>>=20 >>>>>>>> Oleg >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>> On Thu, Aug 20, 2015 at 7:55 PM, Oleg Kalnichevski < >> olegk@apache.org >>>>>>>=20 >>>>>>>> wrote: >>>>>>>>>=20 >>>>>>>>>>> On Thu, 2015-08-20 at 19:55 +0530, Shushant Arora wrote: >>>>>>>>>>> how to avoid this exception - how to handle this. >>>>>>>>>>=20 >>>>>>>>>> Unplugging your computer from the network will reliably eliminate= >>>>>> the >>>>>>>>>> problem. >>>>>>>>>>=20 >>>>>>>>>> Oleg >>>>>>>>>>=20 >>>>>>>>>>> On Thu, Aug 20, 2015 at 7:45 PM, Oleg Kalnichevski < >>>>>> olegk@apache.org >>>>>>>>>=20 >>>>>>>>>> wrote: >>>>>>>>>>>=20 >>>>>>>>>>>>> On Thu, 2015-08-20 at 19:43 +0530, Shushant Arora wrote: >>>>>>>>>>>>> Hi >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I am using HttpAsyncClients and >>>>>>>> PoolingNHttpClientConnectionManager . >>>>>>>>>>>>> Getting below exception when I increase no of http clients on >>>>>>>>>> machine. >>>>>>>>>>>> What >>>>>>>>>>>>> could be the reason and solution for this. >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> org.apache.http.ConnectionClosedException: Connection closed >>>>>>>>>>>>> at >> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyn= cRequestExecutor.java:321) >>>>>>>>>>>>> at >>>>>>>>>>>>=20 >>>>>>>>>>>> The opposite endpoint closed the connection on its end without >>>>>>>>>> correctly >>>>>>>>>>>> completing the message. >>>>>>>>>>>>=20 >>>>>>>>>>>> Oleg >> org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(Defaul= tNHttpClientConnection.java:263) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalI= ODispatch.java:81) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalI= ODispatch.java:39) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIO= Dispatch.java:116) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.jav= a:164) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractI= OReactor.java:339) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(Abstract= IOReactor.java:317) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReac= tor.java:278) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java= :106) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(= AbstractMultiworkerIOReactor.java:590) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.SessionInputBufferImpl.fill(SessionInput= BufferImpl.java:206) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.codecs.AbstractMessageParser.fillBuffer(Abstract= MessageParser.java:138) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(Defaul= tNHttpClientConnection.java:243) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalI= ODispatch.java:81) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalI= ODispatch.java:39) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIO= Dispatch.java:116) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.jav= a:164) >>>>>>>>>>>>> at >> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractI= OReactor.java:3 >> --------------------------------------------------------------------- >>>>>>>>>>>> To unsubscribe, e-mail: >>>>>> httpclient-users-unsubscribe@hc.apache.org >>>>>>>>>>>> For additional commands, e-mail: >>>>>>>> httpclient-users-help@hc.apache.org >>>>>> ---------------------------------------------------------------------= >>>>>>>>>> To unsubscribe, e-mail: >> httpclient-users-unsubscribe@hc.apache.org >>>>>>>>>> For additional commands, e-mail: >>>>>> httpclient-users-help@hc.apache.org >> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org >>>>>>>> For additional commands, e-mail: >> httpclient-users-help@hc.apache.org >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> ---------------------------------------------------------------------= >>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org >>>>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org >>>>=20 >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org >>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org >>=20 >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org >> For additional commands, e-mail: httpclient-users-help@hc.apache.org >>=20 >>=20 --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org For additional commands, e-mail: httpclient-users-help@hc.apache.org