From httpclient-users-return-14258-archive-asf-public=cust-asf.ponee.io@hc.apache.org Mon Jun 11 17:16:42 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id B98D2180647 for ; Mon, 11 Jun 2018 17:16:41 +0200 (CEST) Received: (qmail 46390 invoked by uid 500); 11 Jun 2018 15:16:40 -0000 Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "HttpClient User Discussion" Delivered-To: mailing list httpclient-users@hc.apache.org Received: (qmail 46379 invoked by uid 99); 11 Jun 2018 15:16:40 -0000 Received: from mail-relay.apache.org (HELO mailrelay2-lw-us.apache.org) (207.244.88.137) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jun 2018 15:16:40 +0000 Received: from ok2c (unknown [213.55.211.94]) by mailrelay2-lw-us.apache.org (ASF Mail Server at mailrelay2-lw-us.apache.org) with ESMTPSA id 9D28E2AEF for ; Mon, 11 Jun 2018 15:16:38 +0000 (UTC) Message-ID: <1528730196.18954.8.camel@apache.org> Subject: Re: HttpClient thread seems waiting for stuff for a long time and do not stop From: Oleg Kalnichevski To: HttpClient User Discussion Date: Mon, 11 Jun 2018 17:16:36 +0200 In-Reply-To: References: <72414B77-85C7-4FAF-98B5-A3B3FABFF93C@163.com> <1528720068.13204.1.camel@apache.org> <1528727220.18954.2.camel@apache.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6-1+deb9u1 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit On Mon, 2018-06-11 at 23:13 +0800, zhangminglei wrote: > Hi, Oleg. > > I found the interesting thing. If I change the code like the below, > everything will be fine. No, it will not. Your code keeps on leaking connections. Please fix it. Oleg > I just make the url different from one of them. Url, Url1, Url2 . > This time, the program can stoped. Confused….. > > Minglei. > > public class HttpClientWithSync { >    public static void main(String[] args) { > >       String url = "https://www.baidu.com/"; > >       String url1 = "http://hc.apache.org/httpcomponents-client-4.5.x > /tutorial/html/fundamentals.html"; > >       String url2 = "http://news.baidu.com/"; > >       String[] strs = new String[] {url, url1, url2}; > >       CloseableHttpClient httpclient = HttpClients.createDefault(); > >       for (int i = 0; i < 3; i++) { >          HttpGet httpget = new HttpGet(strs[i]); >          System.out.println(); >          try { >             System.out.println("Waiting for the contents, hurry! > hurry ! hurry!....."); > >             // take a while and block the main thread. >             HttpResponse response = httpclient.execute(httpget); > >             if (response.getStatusLine().getStatusCode() == > HttpStatus.SC_OK) { >                System.out.println("============ Get the entity > ============ "+ response.getEntity()); >             } >             // after getting the value from the url and then do this > or do not if something unexpected to happen. >             System.out.println("Do some other stuff"); >          } catch (IOException e) { >             e.printStackTrace(); >          } >       } >       try { >          httpclient.close(); >       } catch (IOException e) { >          e.printStackTrace(); >       } >    } > } > > > > 在 2018年6月11日，下午10:27，Oleg Kalnichevski 写道： > > > > On Mon, 2018-06-11 at 21:12 +0800, zhangminglei wrote: > > > Hi, Oleg, > > > > > > Thank you for your response, But my question is why this program > > > never stopped ? And always running… > > > > > > > The first two requests cause all available connections to get > > leaked. > > All subsequent requests block indefinitely waiting for connections > > to > > become available in the connection pool.  > > > > Oleg > > > > > Minglei. > > > > > > > 在 2018年6月11日，下午8:27，Oleg Kalnichevski 写道： > > > > > > > > On Fri, 2018-06-08 at 15:30 +0800, zhangminglei wrote: > > > > > Hi, friends. > > > > > > > > > >  I am using HttpClient for a few tests. And I use the below > > > > > code > > > > > to > > > > > run without success. But the process never stoped.  > > > > > I use jstack for this and found it always wait for something. > > > > > Could > > > > > you help me please ? Thanks. > > > > > > > > > > parking to wait for <0x000000076c1d0980> (a > > > > > java.util.concurrent.locks.AbstractQueuedSynchronizer$Conditi > > > > > onOb > > > > > ject > > > > > ) > > > > > > > > > > Best > > > > > Minglei > > > > > > > > > > public class HttpClientWithSync { > > > > > > > > > >    public static void main(String[] args) { > > > > > > > > > >       String url = "https://www.baidu.com/"; > > > > > > > > > >       String[] strs = new String[] {url, url, url}; > > > > > > > > > >       CloseableHttpClient httpclient = > > > > > HttpClients.createDefault(); > > > > > > > > > >       for (int i = 0; i < 3; i++) { > > > > >          HttpGet httpget = new HttpGet(strs[i]); > > > > >          System.out.println(); > > > > >          try { > > > > >             System.out.println("Waiting for the contents, > > > > > hurry! > > > > > hurry ! hurry!....."); > > > > > > > > > >             // take a while and block the main thread. > > > > >             HttpResponse response = > > > > > httpclient.execute(httpget); > > > > > > > > > >             if (response.getStatusLine().getStatusCode() == > > > > > HttpStatus.SC_OK) { > > > > >                System.out.println("============ Get the > > > > > entity > > > > > ============ "+ response.getEntity()); > > > > >             } > > > > >             // after getting the value from the url and then > > > > > do > > > > > this > > > > > or do not if something unexpected to happen. > > > > >             System.out.println("Do some other stuff"); > > > > >          } catch (IOException e) { > > > > >             e.printStackTrace(); > > > > >          } > > > > >       } > > > > >       try { > > > > >          httpclient.close(); > > > > >       } catch (IOException e) { > > > > >          e.printStackTrace(); > > > > >       } > > > > >    } > > > > > } > > > > > > > > > > > > > Your code is leaking connections. > > > > > > > > Please see  > > > > > > > > http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/ > > > > fund > > > > amen > > > 4.5.x/tutorial/html/fundamen> > > > > tals.html#d5e145 > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > The full thread dump are > > > > > > > > > > 2018-06-08 11:02:08 > > > > > Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.161- > > > > > b12 > > > > > mixed > > > > > mode): > > > > > > > > > > "Attach Listener" #12 daemon prio=9 os_prio=31 > > > > > tid=0x00007fce8904a000 > > > > > nid=0x1207 waiting on condition [0x0000000000000000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > > > > > > > "Service Thread" #10 daemon prio=9 os_prio=31 > > > > > tid=0x00007fce8c04b800 > > > > > nid=0x5503 runnable [0x0000000000000000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > > > > > > > "C1 CompilerThread3" #9 daemon prio=9 os_prio=31 > > > > > tid=0x00007fce8c002800 nid=0x5303 waiting on condition > > > > > [0x0000000000000000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > > > > > > > "C2 CompilerThread2" #8 daemon prio=9 os_prio=31 > > > > > tid=0x00007fce89084000 nid=0x5103 waiting on condition > > > > > [0x0000000000000000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > > > > > > > "C2 CompilerThread1" #7 daemon prio=9 os_prio=31 > > > > > tid=0x00007fce8c002000 nid=0x4f03 waiting on condition > > > > > [0x0000000000000000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > > > > > > > "C2 CompilerThread0" #6 daemon prio=9 os_prio=31 > > > > > tid=0x00007fce8c001000 nid=0x4d03 waiting on condition > > > > > [0x0000000000000000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > > > > > > > "Monitor Ctrl-Break" #5 daemon prio=5 os_prio=31 > > > > > tid=0x00007fce8a83b800 nid=0x4b03 runnable > > > > > [0x0000700007e38000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > >    at java.net.SocketInputStream.socketRead0(Native Method) > > > > >    at > > > > > java.net.SocketInputStream.socketRead(SocketInputStream.java: > > > > > 116) > > > > >    at > > > > > java.net.SocketInputStream.read(SocketInputStream.java:171) > > > > >    at > > > > > java.net.SocketInputStream.read(SocketInputStream.java:141) > > > > >    at > > > > > sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) > > > > >    at > > > > > sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) > > > > >    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) > > > > >    - locked <0x000000076adce880> (a > > > > > java.io.InputStreamReader) > > > > >    at > > > > > java.io.InputStreamReader.read(InputStreamReader.java:184) > > > > >    at java.io.BufferedReader.fill(BufferedReader.java:161) > > > > >    at > > > > > java.io.BufferedReader.readLine(BufferedReader.java:324) > > > > >    - locked <0x000000076adce880> (a > > > > > java.io.InputStreamReader) > > > > >    at > > > > > java.io.BufferedReader.readLine(BufferedReader.java:389) > > > > >    at > > > > > com.intellij.rt.execution.application.AppMainV2$1.run(AppMain > > > > > V2.j > > > > > ava: > > > > > 64) > > > > > > > > > > "Signal Dispatcher" #4 daemon prio=9 os_prio=31 > > > > > tid=0x00007fce8881c000 nid=0x4903 runnable > > > > > [0x0000000000000000] > > > > >    java.lang.Thread.State: RUNNABLE > > > > > > > > > > "Finalizer" #3 daemon prio=8 os_prio=31 > > > > > tid=0x00007fce89003800 > > > > > nid=0x3903 in Object.wait() [0x0000700007c32000] > > > > >    java.lang.Thread.State: WAITING (on object monitor) > > > > >    at java.lang.Object.wait(Native Method) > > > > >    - waiting on <0x000000076ab08ec0> (a > > > > > java.lang.ref.ReferenceQueue$Lock) > > > > >    at > > > > > java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) > > > > >    - locked <0x000000076ab08ec0> (a > > > > > java.lang.ref.ReferenceQueue$Lock) > > > > >    at > > > > > java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) > > > > >    at > > > > > java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:20 > > > > > 9) > > > > > > > > > > "Reference Handler" #2 daemon prio=10 os_prio=31 > > > > > tid=0x00007fce8b004000 nid=0x3703 in Object.wait() > > > > > [0x0000700007b2f000] > > > > >    java.lang.Thread.State: WAITING (on object monitor) > > > > >    at java.lang.Object.wait(Native Method) > > > > >    - waiting on <0x000000076ab06b68> (a > > > > > java.lang.ref.Reference$Lock) > > > > >    at java.lang.Object.wait(Object.java:502) > > > > >    at > > > > > java.lang.ref.Reference.tryHandlePending(Reference.java:191) > > > > >    - locked <0x000000076ab06b68> (a > > > > > java.lang.ref.Reference$Lock) > > > > >    at > > > > > java.lang.ref.Reference$ReferenceHandler.run(Reference.java:1 > > > > > 53) > > > > > > > > > > "main" #1 prio=5 os_prio=31 tid=0x00007fce8a008000 nid=0x1c03 > > > > > waiting > > > > > on condition [0x0000700007110000] > > > > >    java.lang.Thread.State: WAITING (parking) > > > > >    at sun.misc.Unsafe.park(Native Method) > > > > >    - parking to wait for  <0x000000076c1d0980> (a > > > > > java.util.concurrent.locks.AbstractQueuedSynchronizer$Conditi > > > > > onOb > > > > > ject > > > > > ) > > > > >    at > > > > > java.util.concurrent.locks.LockSupport.park(LockSupport.java: > > > > > 175) > > > > >    at > > > > > java.util.concurrent.locks.AbstractQueuedSynchronizer$Conditi > > > > > onOb > > > > > ject > > > > > .await(AbstractQueuedSynchronizer.java:2039) > > > > >    at > > > > > org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(Ab > > > > > stra > > > > > ctCo > > > > > nnPool.java:377) > > > > >    at > > > > > org.apache.http.pool.AbstractConnPool.access$200(AbstractConn > > > > > Pool > > > > > .jav > > > > > a:67) > > > > >    at > > > > > org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool. > > > > > java > > > > > :243 > > > > > ) > > > > >    - locked <0x000000076d160008> (a > > > > > org.apache.http.pool.AbstractConnPool$2) > > > > >    at > > > > > org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool. > > > > > java > > > > > :191 > > > > > ) > > > > >    at > > > > > org.apache.http.impl.conn.PoolingHttpClientConnectionManager. > > > > > leas > > > > > eCon > > > > > nection(PoolingHttpClientConnectionManager.java:282) > > > > >    at > > > > > org.apache.http.impl.conn.PoolingHttpClientConnectionManager$ > > > > > 1.ge > > > > > t(Po > > > > > olingHttpClientConnectionManager.java:269) > > > > >    at > > > > > org.apache.http.impl.execchain.MainClientExec.execute(MainCli > > > > > entE > > > > > xec. > > > > > java:191) > > > > >    at > > > > > org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolE > > > > > xec. > > > > > java > > > > > :185) > > > > >    at > > > > > org.apache.http.impl.execchain.RetryExec.execute(RetryExec.ja > > > > > va:8 > > > > > 9) > > > > >    at > > > > > org.apache.http.impl.execchain.RedirectExec.execute(RedirectE > > > > > xec. > > > > > java > > > > > :111) > > > > >    at > > > > > org.apache.http.impl.client.InternalHttpClient.doExecute(Inte > > > > > rnal > > > > > Http > > > > > Client.java:185) > > > > >    at > > > > > org.apache.http.impl.client.CloseableHttpClient.execute(Close > > > > > able > > > > > Http > > > > > Client.java:83) > > > > >    at > > > > > org.apache.http.impl.client.CloseableHttpClient.execute(Close > > > > > able > > > > > Http > > > > > Client.java:108) > > > > >    at > > > > > org.apache.flink.streaming.connectors.fs.HttpClientWithSync.m > > > > > ain( > > > > > Http > > > > > ClientWithSync.java:37) > > > > > > > > > > "VM Thread" os_prio=31 tid=0x00007fce88819000 nid=0x3503 > > > > > runnable  > > > > > > > > > > "GC task thread#0 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce8a014000 > > > > > nid=0x2503 runnable  > > > > > > > > > > "GC task thread#1 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce8a014800 > > > > > nid=0x2703 runnable  > > > > > > > > > > "GC task thread#2 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce89004800 > > > > > nid=0x2903 runnable  > > > > > > > > > > "GC task thread#3 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce88800800 > > > > > nid=0x2b03 runnable  > > > > > > > > > > "GC task thread#4 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce88808800 > > > > > nid=0x2d03 runnable  > > > > > > > > > > "GC task thread#5 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce88809000 > > > > > nid=0x2f03 runnable  > > > > > > > > > > "GC task thread#6 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce89005800 > > > > > nid=0x3103 runnable  > > > > > > > > > > "GC task thread#7 (ParallelGC)" os_prio=31 > > > > > tid=0x00007fce89006000 > > > > > nid=0x3303 runnable  > > > > > > > > > > "VM Periodic Task Thread" os_prio=31 tid=0x00007fce888c7000 > > > > > nid=0x5703 waiting on condition  > > > > > > > > > > JNI global references: 364 > > > > > > > > > > > > > > > > > > ------------------------------------------------------------- > > > > ---- > > > > ---- > > > > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache. > > > > org > > > > > > > > For additional commands, e-mail: httpclient-users-help@hc.apach > > > > e.or > > > > g > > > > ----------------------------------------------------------------- > > ---- > > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org > > > > For additional commands, e-mail: httpclient-users-help@hc.apache.or > > g --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org For additional commands, e-mail: httpclient-users-help@hc.apache.org