From httpclient-users-return-14257-archive-asf-public=cust-asf.ponee.io@hc.apache.org Mon Jun 11 17:14:21 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 22584180647 for ; Mon, 11 Jun 2018 17:14:20 +0200 (CEST) Received: (qmail 42004 invoked by uid 500); 11 Jun 2018 15:14:20 -0000 Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "HttpClient User Discussion" Delivered-To: mailing list httpclient-users@hc.apache.org Received: (qmail 41992 invoked by uid 99); 11 Jun 2018 15:14:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jun 2018 15:14:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id C4C6E182E6D for ; Mon, 11 Jun 2018 15:14:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.471 X-Spam-Level: ** X-Spam-Status: No, score=2.471 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FROM_LOCAL_DIGITS=0.001, FROM_LOCAL_HEX=0.331, HTML_MESSAGE=2, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=163.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 6IDu-hq1VWe6 for ; Mon, 11 Jun 2018 15:14:14 +0000 (UTC) Received: from m50-134.163.com (m50-134.163.com [123.125.50.134]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id A5DB45F359 for ; Mon, 11 Jun 2018 15:14:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:Message-Id:Mime-Version:Subject:Date; bh=DS2Lg KbYWBP6finpIWalM970i0UZ75PWYsKav8eTc7I=; b=hYagI4Mffn3i/ugcKcbi5 jC02p88Kqbj1Z0nnlKqR/1xEiDFo5PN7AfPaq2Eln7JgZesjCj1Rflk42T5GPu2a eXQRIyUkP+0XE+bBph4KNCikwPmZcOJlc7cemC+0xc/L1+lhW2wEoLlqKZ61pRt6 kS3thh9v9bsTtaTS3Gf3A4= Received: from [10.107.102.139] (unknown [112.65.125.218]) by smtp4 (Coremail) with SMTP id DtGowAD39ZK3kR5b1NRJAA--.13935S3; Mon, 11 Jun 2018 23:14:00 +0800 (CST) From: zhangminglei <18717838093@163.com> Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_BE4239D1-805B-4E18-8861-FD1F6158B02C" Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: HttpClient thread seems waiting for stuff for a long time and do not stop Date: Mon, 11 Jun 2018 23:13:56 +0800 In-Reply-To: <1528727220.18954.2.camel@apache.org> Cc: Oleg Kalnichevski To: HttpClient User Discussion References: <72414B77-85C7-4FAF-98B5-A3B3FABFF93C@163.com> <1528720068.13204.1.camel@apache.org> <1528727220.18954.2.camel@apache.org> X-Mailer: Apple Mail (2.3259) X-CM-TRANSID:DtGowAD39ZK3kR5b1NRJAA--.13935S3 X-Coremail-Antispam: 1Uf129KBjvJXoW3tF1kKr4DGFW3uFyUtFWUArb_yoWDKF45pr 4akr1fKF48ZrZxZrs7X3W0yr1avrW0vFWDG3ykGF1vva90yrs0q3WjyrWF9ry8CFWUZa1j q3WjvF1IqrsrA3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07UEks9UUUUU= X-Originating-IP: [112.65.125.218] X-CM-SenderInfo: 52poijayvwimiwqxiqqrwthudrp/1tbiTRhiNlc7DMa4bAAAsz --Apple-Mail=_BE4239D1-805B-4E18-8861-FD1F6158B02C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=gb2312 Hi, Oleg. I found the interesting thing. If I change the code like the below, = everything will be fine. I just make the url different from one of them. = Url, Url1, Url2 . This time, the program can stoped. Confused=A1=AD.. Minglei. public class HttpClientWithSync { public static void main(String[] args) { String url =3D "https://www.baidu.com/"; String url1 =3D = "http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/fundamenta= ls.html"; String url2 =3D "http://news.baidu.com/"; String[] strs =3D new String[] {url, url1, url2}; CloseableHttpClient httpclient =3D HttpClients.createDefault(); for (int i =3D 0; i < 3; i++) { HttpGet httpget =3D new HttpGet(strs[i]); System.out.println(); try { System.out.println("Waiting for the contents, hurry! hurry ! = hurry!....."); // take a while and block the main thread. HttpResponse response =3D httpclient.execute(httpget); if (response.getStatusLine().getStatusCode() =3D=3D = HttpStatus.SC_OK) { System.out.println("=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D = Get the entity =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D "+ = response.getEntity()); } // after getting the value from the url and then do this or = do not if something unexpected to happen. System.out.println("Do some other stuff"); } catch (IOException e) { e.printStackTrace(); } } try { httpclient.close(); } catch (IOException e) { e.printStackTrace(); } } } > =D4=DA 2018=C4=EA6=D4=C211=C8=D5=A3=AC=CF=C2=CE=E710:27=A3=ACOleg = Kalnichevski =D0=B4=B5=C0=A3=BA >=20 > On Mon, 2018-06-11 at 21:12 +0800, zhangminglei wrote: >> Hi, Oleg, >>=20 >> Thank you for your response, But my question is why this program >> never stopped ? And always running=A1=AD >>=20 >=20 > The first two requests cause all available connections to get leaked. > All subsequent requests block indefinitely waiting for connections to > become available in the connection pool.=20 >=20 > Oleg >=20 >> Minglei. >>=20 >>> =D4=DA 2018=C4=EA6=D4=C211=C8=D5=A3=AC=CF=C2=CE=E78:27=A3=ACOleg = Kalnichevski =D0=B4=B5=C0=A3=BA >>>=20 >>> On Fri, 2018-06-08 at 15:30 +0800, zhangminglei wrote: >>>> Hi, friends. >>>>=20 >>>> I am using HttpClient for a few tests. And I use the below code >>>> to >>>> run without success. But the process never stoped.=20 >>>> I use jstack for this and found it always wait for something. >>>> Could >>>> you help me please ? Thanks. >>>>=20 >>>> parking to wait for <0x000000076c1d0980> (a >>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionOb >>>> ject >>>> ) >>>>=20 >>>> Best >>>> Minglei >>>>=20 >>>> public class HttpClientWithSync { >>>>=20 >>>> public static void main(String[] args) { >>>>=20 >>>> String url =3D "https://www.baidu.com/"; >>>>=20 >>>> String[] strs =3D new String[] {url, url, url}; >>>>=20 >>>> CloseableHttpClient httpclient =3D >>>> HttpClients.createDefault(); >>>>=20 >>>> for (int i =3D 0; i < 3; i++) { >>>> HttpGet httpget =3D new HttpGet(strs[i]); >>>> System.out.println(); >>>> try { >>>> System.out.println("Waiting for the contents, hurry! >>>> hurry ! hurry!....."); >>>>=20 >>>> // take a while and block the main thread. >>>> HttpResponse response =3D httpclient.execute(httpget); >>>>=20 >>>> if (response.getStatusLine().getStatusCode() =3D=3D >>>> HttpStatus.SC_OK) { >>>> System.out.println("=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= Get the entity >>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D "+ response.getEntity()); >>>> } >>>> // after getting the value from the url and then do >>>> this >>>> or do not if something unexpected to happen. >>>> System.out.println("Do some other stuff"); >>>> } catch (IOException e) { >>>> e.printStackTrace(); >>>> } >>>> } >>>> try { >>>> httpclient.close(); >>>> } catch (IOException e) { >>>> e.printStackTrace(); >>>> } >>>> } >>>> } >>>>=20 >>>=20 >>> Your code is leaking connections. >>>=20 >>> Please see=20 >>>=20 >>> http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/fund >>> amen >> 4.5.x/tutorial/html/fundamen> >>> tals.html#d5e145 >>>=20 >>> Oleg >>>=20 >>>=20 >>>>=20 >>>> The full thread dump are >>>>=20 >>>> 2018-06-08 11:02:08 >>>> Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.161-b12 >>>> mixed >>>> mode): >>>>=20 >>>> "Attach Listener" #12 daemon prio=3D9 os_prio=3D31 >>>> tid=3D0x00007fce8904a000 >>>> nid=3D0x1207 waiting on condition [0x0000000000000000] >>>> java.lang.Thread.State: RUNNABLE >>>>=20 >>>> "Service Thread" #10 daemon prio=3D9 os_prio=3D31 >>>> tid=3D0x00007fce8c04b800 >>>> nid=3D0x5503 runnable [0x0000000000000000] >>>> java.lang.Thread.State: RUNNABLE >>>>=20 >>>> "C1 CompilerThread3" #9 daemon prio=3D9 os_prio=3D31 >>>> tid=3D0x00007fce8c002800 nid=3D0x5303 waiting on condition >>>> [0x0000000000000000] >>>> java.lang.Thread.State: RUNNABLE >>>>=20 >>>> "C2 CompilerThread2" #8 daemon prio=3D9 os_prio=3D31 >>>> tid=3D0x00007fce89084000 nid=3D0x5103 waiting on condition >>>> [0x0000000000000000] >>>> java.lang.Thread.State: RUNNABLE >>>>=20 >>>> "C2 CompilerThread1" #7 daemon prio=3D9 os_prio=3D31 >>>> tid=3D0x00007fce8c002000 nid=3D0x4f03 waiting on condition >>>> [0x0000000000000000] >>>> java.lang.Thread.State: RUNNABLE >>>>=20 >>>> "C2 CompilerThread0" #6 daemon prio=3D9 os_prio=3D31 >>>> tid=3D0x00007fce8c001000 nid=3D0x4d03 waiting on condition >>>> [0x0000000000000000] >>>> java.lang.Thread.State: RUNNABLE >>>>=20 >>>> "Monitor Ctrl-Break" #5 daemon prio=3D5 os_prio=3D31 >>>> tid=3D0x00007fce8a83b800 nid=3D0x4b03 runnable [0x0000700007e38000] >>>> java.lang.Thread.State: RUNNABLE >>>> at java.net.SocketInputStream.socketRead0(Native Method) >>>> at >>>> java.net.SocketInputStream.socketRead(SocketInputStream.java:116) >>>> at java.net.SocketInputStream.read(SocketInputStream.java:171) >>>> at java.net.SocketInputStream.read(SocketInputStream.java:141) >>>> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) >>>> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) >>>> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) >>>> - locked <0x000000076adce880> (a java.io.InputStreamReader) >>>> at java.io.InputStreamReader.read(InputStreamReader.java:184) >>>> at java.io.BufferedReader.fill(BufferedReader.java:161) >>>> at java.io.BufferedReader.readLine(BufferedReader.java:324) >>>> - locked <0x000000076adce880> (a java.io.InputStreamReader) >>>> at java.io.BufferedReader.readLine(BufferedReader.java:389) >>>> at >>>> com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.j >>>> ava: >>>> 64) >>>>=20 >>>> "Signal Dispatcher" #4 daemon prio=3D9 os_prio=3D31 >>>> tid=3D0x00007fce8881c000 nid=3D0x4903 runnable [0x0000000000000000] >>>> java.lang.Thread.State: RUNNABLE >>>>=20 >>>> "Finalizer" #3 daemon prio=3D8 os_prio=3D31 tid=3D0x00007fce89003800 >>>> nid=3D0x3903 in Object.wait() [0x0000700007c32000] >>>> java.lang.Thread.State: WAITING (on object monitor) >>>> at java.lang.Object.wait(Native Method) >>>> - waiting on <0x000000076ab08ec0> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> at >>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) >>>> - locked <0x000000076ab08ec0> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> at >>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) >>>> at >>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>>>=20 >>>> "Reference Handler" #2 daemon prio=3D10 os_prio=3D31 >>>> tid=3D0x00007fce8b004000 nid=3D0x3703 in Object.wait() >>>> [0x0000700007b2f000] >>>> java.lang.Thread.State: WAITING (on object monitor) >>>> at java.lang.Object.wait(Native Method) >>>> - waiting on <0x000000076ab06b68> (a >>>> java.lang.ref.Reference$Lock) >>>> at java.lang.Object.wait(Object.java:502) >>>> at >>>> java.lang.ref.Reference.tryHandlePending(Reference.java:191) >>>> - locked <0x000000076ab06b68> (a java.lang.ref.Reference$Lock) >>>> at >>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) >>>>=20 >>>> "main" #1 prio=3D5 os_prio=3D31 tid=3D0x00007fce8a008000 nid=3D0x1c03= >>>> waiting >>>> on condition [0x0000700007110000] >>>> java.lang.Thread.State: WAITING (parking) >>>> at sun.misc.Unsafe.park(Native Method) >>>> - parking to wait for <0x000000076c1d0980> (a >>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionOb >>>> ject >>>> ) >>>> at >>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) >>>> at >>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionOb >>>> ject >>>> .await(AbstractQueuedSynchronizer.java:2039) >>>> at >>>> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(Abstra >>>> ctCo >>>> nnPool.java:377) >>>> at >>>> org.apache.http.pool.AbstractConnPool.access$200(AbstractConnPool >>>> .jav >>>> a:67) >>>> at >>>> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java >>>> :243 >>>> ) >>>> - locked <0x000000076d160008> (a >>>> org.apache.http.pool.AbstractConnPool$2) >>>> at >>>> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.java >>>> :191 >>>> ) >>>> at >>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leas >>>> eCon >>>> nection(PoolingHttpClientConnectionManager.java:282) >>>> at >>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.ge >>>> t(Po >>>> olingHttpClientConnectionManager.java:269) >>>> at >>>> org.apache.http.impl.execchain.MainClientExec.execute(MainClientE >>>> xec. >>>> java:191) >>>> at >>>> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec. >>>> java >>>> :185) >>>> at >>>> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:8 >>>> 9) >>>> at >>>> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec. >>>> java >>>> :111) >>>> at >>>> org.apache.http.impl.client.InternalHttpClient.doExecute(Internal >>>> Http >>>> Client.java:185) >>>> at >>>> org.apache.http.impl.client.CloseableHttpClient.execute(Closeable >>>> Http >>>> Client.java:83) >>>> at >>>> org.apache.http.impl.client.CloseableHttpClient.execute(Closeable >>>> Http >>>> Client.java:108) >>>> at >>>> org.apache.flink.streaming.connectors.fs.HttpClientWithSync.main( >>>> Http >>>> ClientWithSync.java:37) >>>>=20 >>>> "VM Thread" os_prio=3D31 tid=3D0x00007fce88819000 nid=3D0x3503 >>>> runnable=20 >>>>=20 >>>> "GC task thread#0 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce8a014000= >>>> nid=3D0x2503 runnable=20 >>>>=20 >>>> "GC task thread#1 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce8a014800= >>>> nid=3D0x2703 runnable=20 >>>>=20 >>>> "GC task thread#2 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce89004800= >>>> nid=3D0x2903 runnable=20 >>>>=20 >>>> "GC task thread#3 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce88800800= >>>> nid=3D0x2b03 runnable=20 >>>>=20 >>>> "GC task thread#4 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce88808800= >>>> nid=3D0x2d03 runnable=20 >>>>=20 >>>> "GC task thread#5 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce88809000= >>>> nid=3D0x2f03 runnable=20 >>>>=20 >>>> "GC task thread#6 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce89005800= >>>> nid=3D0x3103 runnable=20 >>>>=20 >>>> "GC task thread#7 (ParallelGC)" os_prio=3D31 tid=3D0x00007fce89006000= >>>> nid=3D0x3303 runnable=20 >>>>=20 >>>> "VM Periodic Task Thread" os_prio=3D31 tid=3D0x00007fce888c7000 >>>> nid=3D0x5703 waiting on condition=20 >>>>=20 >>>> JNI global references: 364 >>>>=20 >>>>=20 >>>=20 >>> ----------------------------------------------------------------- >>> ---- >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org >>> >>> For additional commands, e-mail: httpclient-users-help@hc.apache.or >>> g >=20 > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org = > For additional commands, e-mail: httpclient-users-help@hc.apache.org = --Apple-Mail=_BE4239D1-805B-4E18-8861-FD1F6158B02C--