hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zhangminglei <18717838...@163.com>
Subject Re: HttpClient thread seems waiting for stuff for a long time and do not stop
Date Mon, 11 Jun 2018 15:26:59 GMT
Yes. I know it is leaking connections…. But it can stop the program in the end. If I make
the url the same, it can not stop….

> 在 2018年6月11日,下午11:16,Oleg Kalnichevski <olegk@apache.org> 写道:
> 
> On Mon, 2018-06-11 at 23:13 +0800, zhangminglei wrote:
>> Hi, Oleg.
>> 
>> I found the interesting thing. If I change the code like the below,
>> everything will be fine.
> 
> No, it will not. Your code keeps on leaking connections. Please fix it.
> 
> Oleg 
> 
> 
>> I just make the url different from one of them. Url, Url1, Url2 .
>> This time, the program can stoped. Confused…..
>> 
>> Minglei.
>> 
>> public class HttpClientWithSync {
>>    public static void main(String[] args) {
>> 
>>       String url = "https://www.baidu.com/";
>> 
>>       String url1 = "http://hc.apache.org/httpcomponents-client-4.5.x
>> /tutorial/html/fundamentals.html";
>> 
>>       String url2 = "http://news.baidu.com/";
>> 
>>       String[] strs = new String[] {url, url1, url2};
>> 
>>       CloseableHttpClient httpclient = HttpClients.createDefault();
>> 
>>       for (int i = 0; i < 3; i++) {
>>          HttpGet httpget = new HttpGet(strs[i]);
>>          System.out.println();
>>          try {
>>             System.out.println("Waiting for the contents, hurry!
>> hurry ! hurry!.....");
>> 
>>             // take a while and block the main thread.
>>             HttpResponse response = httpclient.execute(httpget);
>> 
>>             if (response.getStatusLine().getStatusCode() ==
>> HttpStatus.SC_OK) {
>>                System.out.println("============ Get the entity
>> ============ "+ response.getEntity());
>>             }
>>             // after getting the value from the url and then do this
>> or do not if something unexpected to happen.
>>             System.out.println("Do some other stuff");
>>          } catch (IOException e) {
>>             e.printStackTrace();
>>          }
>>       }
>>       try {
>>          httpclient.close();
>>       } catch (IOException e) {
>>          e.printStackTrace();
>>       }
>>    }
>> }
>> 
>> 
>>> 在 2018年6月11日,下午10:27,Oleg Kalnichevski <olegk@apache.org>
写道:
>>> 
>>> On Mon, 2018-06-11 at 21:12 +0800, zhangminglei wrote:
>>>> Hi, Oleg,
>>>> 
>>>> Thank you for your response, But my question is why this program
>>>> never stopped ? And always running…
>>>> 
>>> 
>>> The first two requests cause all available connections to get
>>> leaked.
>>> All subsequent requests block indefinitely waiting for connections
>>> to
>>> become available in the connection pool. 
>>> 
>>> Oleg
>>> 
>>>> Minglei.
>>>> 
>>>>> 在 2018年6月11日,下午8:27,Oleg Kalnichevski <olegk@apache.org>
写道:
>>>>> 
>>>>> On Fri, 2018-06-08 at 15:30 +0800, zhangminglei wrote:
>>>>>> Hi, friends.
>>>>>> 
>>>>>>  I am using HttpClient for a few tests. And I use the below
>>>>>> code
>>>>>> to
>>>>>> run without success. But the process never stoped. 
>>>>>> I use jstack for this and found it always wait for something.
>>>>>> Could
>>>>>> you help me please ? Thanks.
>>>>>> 
>>>>>> parking to wait for <0x000000076c1d0980> (a
>>>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$Conditi
>>>>>> onOb
>>>>>> ject
>>>>>> )
>>>>>> 
>>>>>> Best
>>>>>> Minglei
>>>>>> 
>>>>>> public class HttpClientWithSync {
>>>>>> 
>>>>>>    public static void main(String[] args) {
>>>>>> 
>>>>>>       String url = "https://www.baidu.com/";
>>>>>> 
>>>>>>       String[] strs = new String[] {url, url, url};
>>>>>> 
>>>>>>       CloseableHttpClient httpclient =
>>>>>> HttpClients.createDefault();
>>>>>> 
>>>>>>       for (int i = 0; i < 3; i++) {
>>>>>>          HttpGet httpget = new HttpGet(strs[i]);
>>>>>>          System.out.println();
>>>>>>          try {
>>>>>>             System.out.println("Waiting for the contents,
>>>>>> hurry!
>>>>>> hurry ! hurry!.....");
>>>>>> 
>>>>>>             // take a while and block the main thread.
>>>>>>             HttpResponse response =
>>>>>> httpclient.execute(httpget);
>>>>>> 
>>>>>>             if (response.getStatusLine().getStatusCode() ==
>>>>>> HttpStatus.SC_OK) {
>>>>>>                System.out.println("============ Get the
>>>>>> entity
>>>>>> ============ "+ response.getEntity());
>>>>>>             }
>>>>>>             // after getting the value from the url and then
>>>>>> do
>>>>>> this
>>>>>> or do not if something unexpected to happen.
>>>>>>             System.out.println("Do some other stuff");
>>>>>>          } catch (IOException e) {
>>>>>>             e.printStackTrace();
>>>>>>          }
>>>>>>       }
>>>>>>       try {
>>>>>>          httpclient.close();
>>>>>>       } catch (IOException e) {
>>>>>>          e.printStackTrace();
>>>>>>       }
>>>>>>    }
>>>>>> }
>>>>>> 
>>>>> 
>>>>> Your code is leaking connections.
>>>>> 
>>>>> Please see 
>>>>> 
>>>>> http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/
>>>>> fund
>>>>> amen <http://hc.apache.org/httpcomponents-client-
>>>>> 4.5.x/tutorial/html/fundamen>
>>>>> tals.html#d5e145
>>>>> 
>>>>> Oleg
>>>>> 
>>>>> 
>>>>>> 
>>>>>> The full thread dump are
>>>>>> 
>>>>>> 2018-06-08 11:02:08
>>>>>> Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.161-
>>>>>> b12
>>>>>> mixed
>>>>>> mode):
>>>>>> 
>>>>>> "Attach Listener" #12 daemon prio=9 os_prio=31
>>>>>> tid=0x00007fce8904a000
>>>>>> nid=0x1207 waiting on condition [0x0000000000000000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>> 
>>>>>> "Service Thread" #10 daemon prio=9 os_prio=31
>>>>>> tid=0x00007fce8c04b800
>>>>>> nid=0x5503 runnable [0x0000000000000000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>> 
>>>>>> "C1 CompilerThread3" #9 daemon prio=9 os_prio=31
>>>>>> tid=0x00007fce8c002800 nid=0x5303 waiting on condition
>>>>>> [0x0000000000000000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>> 
>>>>>> "C2 CompilerThread2" #8 daemon prio=9 os_prio=31
>>>>>> tid=0x00007fce89084000 nid=0x5103 waiting on condition
>>>>>> [0x0000000000000000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>> 
>>>>>> "C2 CompilerThread1" #7 daemon prio=9 os_prio=31
>>>>>> tid=0x00007fce8c002000 nid=0x4f03 waiting on condition
>>>>>> [0x0000000000000000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>> 
>>>>>> "C2 CompilerThread0" #6 daemon prio=9 os_prio=31
>>>>>> tid=0x00007fce8c001000 nid=0x4d03 waiting on condition
>>>>>> [0x0000000000000000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>> 
>>>>>> "Monitor Ctrl-Break" #5 daemon prio=5 os_prio=31
>>>>>> tid=0x00007fce8a83b800 nid=0x4b03 runnable
>>>>>> [0x0000700007e38000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>>    at java.net.SocketInputStream.socketRead0(Native Method)
>>>>>>    at
>>>>>> java.net.SocketInputStream.socketRead(SocketInputStream.java:
>>>>>> 116)
>>>>>>    at
>>>>>> java.net.SocketInputStream.read(SocketInputStream.java:171)
>>>>>>    at
>>>>>> java.net.SocketInputStream.read(SocketInputStream.java:141)
>>>>>>    at
>>>>>> sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
>>>>>>    at
>>>>>> sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
>>>>>>    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
>>>>>>    - locked <0x000000076adce880> (a
>>>>>> java.io.InputStreamReader)
>>>>>>    at
>>>>>> java.io.InputStreamReader.read(InputStreamReader.java:184)
>>>>>>    at java.io.BufferedReader.fill(BufferedReader.java:161)
>>>>>>    at
>>>>>> java.io.BufferedReader.readLine(BufferedReader.java:324)
>>>>>>    - locked <0x000000076adce880> (a
>>>>>> java.io.InputStreamReader)
>>>>>>    at
>>>>>> java.io.BufferedReader.readLine(BufferedReader.java:389)
>>>>>>    at
>>>>>> com.intellij.rt.execution.application.AppMainV2$1.run(AppMain
>>>>>> V2.j
>>>>>> ava:
>>>>>> 64)
>>>>>> 
>>>>>> "Signal Dispatcher" #4 daemon prio=9 os_prio=31
>>>>>> tid=0x00007fce8881c000 nid=0x4903 runnable
>>>>>> [0x0000000000000000]
>>>>>>    java.lang.Thread.State: RUNNABLE
>>>>>> 
>>>>>> "Finalizer" #3 daemon prio=8 os_prio=31
>>>>>> tid=0x00007fce89003800
>>>>>> nid=0x3903 in Object.wait() [0x0000700007c32000]
>>>>>>    java.lang.Thread.State: WAITING (on object monitor)
>>>>>>    at java.lang.Object.wait(Native Method)
>>>>>>    - waiting on <0x000000076ab08ec0> (a
>>>>>> java.lang.ref.ReferenceQueue$Lock)
>>>>>>    at
>>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
>>>>>>    - locked <0x000000076ab08ec0> (a
>>>>>> java.lang.ref.ReferenceQueue$Lock)
>>>>>>    at
>>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
>>>>>>    at
>>>>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:20
>>>>>> 9)
>>>>>> 
>>>>>> "Reference Handler" #2 daemon prio=10 os_prio=31
>>>>>> tid=0x00007fce8b004000 nid=0x3703 in Object.wait()
>>>>>> [0x0000700007b2f000]
>>>>>>    java.lang.Thread.State: WAITING (on object monitor)
>>>>>>    at java.lang.Object.wait(Native Method)
>>>>>>    - waiting on <0x000000076ab06b68> (a
>>>>>> java.lang.ref.Reference$Lock)
>>>>>>    at java.lang.Object.wait(Object.java:502)
>>>>>>    at
>>>>>> java.lang.ref.Reference.tryHandlePending(Reference.java:191)
>>>>>>    - locked <0x000000076ab06b68> (a
>>>>>> java.lang.ref.Reference$Lock)
>>>>>>    at
>>>>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:1
>>>>>> 53)
>>>>>> 
>>>>>> "main" #1 prio=5 os_prio=31 tid=0x00007fce8a008000 nid=0x1c03
>>>>>> waiting
>>>>>> on condition [0x0000700007110000]
>>>>>>    java.lang.Thread.State: WAITING (parking)
>>>>>>    at sun.misc.Unsafe.park(Native Method)
>>>>>>    - parking to wait for  <0x000000076c1d0980> (a
>>>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$Conditi
>>>>>> onOb
>>>>>> ject
>>>>>> )
>>>>>>    at
>>>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:
>>>>>> 175)
>>>>>>    at
>>>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$Conditi
>>>>>> onOb
>>>>>> ject
>>>>>> .await(AbstractQueuedSynchronizer.java:2039)
>>>>>>    at
>>>>>> org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(Ab
>>>>>> stra
>>>>>> ctCo
>>>>>> nnPool.java:377)
>>>>>>    at
>>>>>> org.apache.http.pool.AbstractConnPool.access$200(AbstractConn
>>>>>> Pool
>>>>>> .jav
>>>>>> a:67)
>>>>>>    at
>>>>>> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.
>>>>>> java
>>>>>> :243
>>>>>> )
>>>>>>    - locked <0x000000076d160008> (a
>>>>>> org.apache.http.pool.AbstractConnPool$2)
>>>>>>    at
>>>>>> org.apache.http.pool.AbstractConnPool$2.get(AbstractConnPool.
>>>>>> java
>>>>>> :191
>>>>>> )
>>>>>>    at
>>>>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.
>>>>>> leas
>>>>>> eCon
>>>>>> nection(PoolingHttpClientConnectionManager.java:282)
>>>>>>    at
>>>>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager$
>>>>>> 1.ge
>>>>>> t(Po
>>>>>> olingHttpClientConnectionManager.java:269)
>>>>>>    at
>>>>>> org.apache.http.impl.execchain.MainClientExec.execute(MainCli
>>>>>> entE
>>>>>> xec.
>>>>>> java:191)
>>>>>>    at
>>>>>> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolE
>>>>>> xec.
>>>>>> java
>>>>>> :185)
>>>>>>    at
>>>>>> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.ja
>>>>>> va:8
>>>>>> 9)
>>>>>>    at
>>>>>> org.apache.http.impl.execchain.RedirectExec.execute(RedirectE
>>>>>> xec.
>>>>>> java
>>>>>> :111)
>>>>>>    at
>>>>>> org.apache.http.impl.client.InternalHttpClient.doExecute(Inte
>>>>>> rnal
>>>>>> Http
>>>>>> Client.java:185)
>>>>>>    at
>>>>>> org.apache.http.impl.client.CloseableHttpClient.execute(Close
>>>>>> able
>>>>>> Http
>>>>>> Client.java:83)
>>>>>>    at
>>>>>> org.apache.http.impl.client.CloseableHttpClient.execute(Close
>>>>>> able
>>>>>> Http
>>>>>> Client.java:108)
>>>>>>    at
>>>>>> org.apache.flink.streaming.connectors.fs.HttpClientWithSync.m
>>>>>> ain(
>>>>>> Http
>>>>>> ClientWithSync.java:37)
>>>>>> 
>>>>>> "VM Thread" os_prio=31 tid=0x00007fce88819000 nid=0x3503
>>>>>> runnable 
>>>>>> 
>>>>>> "GC task thread#0 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce8a014000
>>>>>> nid=0x2503 runnable 
>>>>>> 
>>>>>> "GC task thread#1 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce8a014800
>>>>>> nid=0x2703 runnable 
>>>>>> 
>>>>>> "GC task thread#2 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce89004800
>>>>>> nid=0x2903 runnable 
>>>>>> 
>>>>>> "GC task thread#3 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce88800800
>>>>>> nid=0x2b03 runnable 
>>>>>> 
>>>>>> "GC task thread#4 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce88808800
>>>>>> nid=0x2d03 runnable 
>>>>>> 
>>>>>> "GC task thread#5 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce88809000
>>>>>> nid=0x2f03 runnable 
>>>>>> 
>>>>>> "GC task thread#6 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce89005800
>>>>>> nid=0x3103 runnable 
>>>>>> 
>>>>>> "GC task thread#7 (ParallelGC)" os_prio=31
>>>>>> tid=0x00007fce89006000
>>>>>> nid=0x3303 runnable 
>>>>>> 
>>>>>> "VM Periodic Task Thread" os_prio=31 tid=0x00007fce888c7000
>>>>>> nid=0x5703 waiting on condition 
>>>>>> 
>>>>>> JNI global references: 364
>>>>>> 
>>>>>> 
>>>>> 
>>>>> -------------------------------------------------------------
>>>>> ----
>>>>> ----
>>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.
>>>>> org
>>>>> <mailto:httpclient-users-unsubscribe@hc.apache.org>
>>>>> For additional commands, e-mail: httpclient-users-help@hc.apach
>>>>> e.or
>>>>> g <mailto:httpclient-users-help@hc.apache.org>
>>> 
>>> -----------------------------------------------------------------
>>> ----
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> <mailto:httpclient-users-unsubscribe@hc.apache.org <mailto:httpclient-users-unsubscribe@hc.apache.org>>
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.or <mailto:httpclient-users-help@hc.apache.or>
>>> g <mailto:httpclient-users-help@hc.apache.org <mailto:httpclient-users-help@hc.apache.org>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org <mailto:httpclient-users-unsubscribe@hc.apache.org>
> For additional commands, e-mail: httpclient-users-help@hc.apache.org <mailto:httpclient-users-help@hc.apache.org>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message