hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajat Sharma" <rsha...@airvananet.com>
Subject RE: Using MultiThreadedHttpConnectionManager results in java.lang.OutOfMemoryError
Date Thu, 15 Dec 2005 16:27:58 GMT
You can also not increase the java heap from default. 

I am sure it is there in jdk1.3 . Try writing java -X on your console and look at the options.
They should be like - 
-Xms<size>        set initial Java heap size
-Xmx<size>        set maximum Java heap size
-Xss<size>        set java thread stack size

Hopefully this is also a very good way to circumvent the outofMemory problem.


Regards,
Rajat


-----Original Message-----
From: Jaya Christina B [mailto:bjaya@mahindrabt.com]
Sent: Tuesday, December 13, 2005 12:36 AM
To: HttpClient User Discussion
Subject: RE: Using MultiThreadedHttpConnectionManager results in
java.lang.OutOfMemoryError



Thanks again Oleg,
We are hitting at a limited number of servers, so I guess we would be
going with the Reuse-HttpClient approach,

Thanks again for your time.


Regards,
Jaya.



-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]

Sent: Monday, December 12, 2005 6:33 PM
To: httpclient-user@jakarta.apache.org
Subject: Re: Using MultiThreadedHttpConnectionManager results in
java.lang.OutOfMemoryError

On Mon, Dec 12, 2005 at 06:29:25PM +0530, Jaya Christina B wrote:
>

> Oleg,
>

> Will reusing the HttpClient object so as to have one per application,

> instead creating one per request help?

It certainly will help, especially if you are hitting just a limited
number of target servers

Oleg


> We could then use MultiThreadedHttpConnectionManager to handle the

> threads?
>

> Thanks a lot for your reply
>

> Regards,
> Jaya.
>

>

> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Monday, December 12, 2005 6:22 PM
> To: httpclient-user@jakarta.apache.org
> Subject: Re: Using MultiThreadedHttpConnectionManager results in

> java.lang.OutOfMemoryError
>

> On Mon, Dec 12, 2005 at 03:45:00PM +0530, Jaya Christina B wrote:
> >
> > Thanks for you response Oleg.
> >
> > We run on JDK1.3 and due to compatibility issues with Cactus VS

> > HttpClient, we for not have to stick with HttpClient v2.0 And we

> > have to use the connection timeouts, as this is one of the main

> > reasons we are HttpClient
> >
>

> Jaya
>

> I am afraid you do not have too many options here. You can consider
> (1) upgrading your JRE from 1.3 to 1.4 and HttpClient from 2.0 to

> 3.0rc4, (2) disabling connection timeout in order to prevent creation

> of controller threads, (3) tweaking OS settings in an attempt to

> increase the maximum number of simultaneous threads supported by the

> JRE, (4) reducing the number of connection (and therefore that of

> controller
> threads) attempts by reusing connections
>

>

> > So, I think these are proving to be some blocks here.
> >
> > I was reading a few more posts on the apache mailing list.
> > The thread is on
> >
> http://mail-archives.apache.org/mod_mbox/jakarta-httpclient-dev/200402
> .m
> > box/%3c401DCB4C.6020506@mensamind.com%3e
> >
> > One of the suggestions were to add a request header to each client

> > request saying 'Connection: close'
> > Would this help us with the problem we are facing?
> > TIA.
> >
>

> This will have no bearing on the 'java.lang.OutOfMemoryError: unable

> to create new native thread' exception.
>

> Oleg
>

>

> >
> > Regards,
> > Jaya
> >
> > -----Original Message-----
> > From: Oleg Kalnichevski [mailto:olegk@apache.org]
> > Sent: Monday, December 12, 2005 3:02 PM
> > To: httpclient-user@jakarta.apache.org
> > Subject: Re: Using MultiThreadedHttpConnectionManager results in

> > java.lang.OutOfMemoryError
> >
> > On Mon, Dec 12, 2005 at 11:05:45AM +0530, Jaya Christina B wrote:
> > >
> > > Hello,
> > >
> > > I'm using HttpClient V2.0 on Sun Solaris/WebLogic7.0
> > >
> > > The code I am using is:-
> > > HttpClient httpClient = new HttpClient(new

> > > MultiThreadedHttpConnectionManager());
> > > httpClient.getHostConfiguration().setHost(host, port); HttpMethod

> > > httpMethod = new GetMethod(url);
> httpClient.executeMethod(httpMethod);
> > > httpMethod.releaseConnection();
> > >
> > > The problem I am facing is that after around 2000 connections,

> > > this results in java.lang.OutOfMemoryError.
> > >
> > > Looks like, even after call is made and finished by calling
> > > releaseConnection() method, the thread does not get finished and
> keeps
> >
> > > on waiting forever.  Hence it creates a thread every time this

> > > functionality is called.
> > >
> > > The weblogic's RequestDispatcher  then throws the exception:
> > > javax.servlet.ServletException: unable to create new native thread
> > >
> > > I have a wrapper class to HttpClient API & create only one
> connection
> > > per HttpClient object, so I think using

> > > MultiThreadedHttpConnectionManager is NOT required.
> > > Please could you have a look at this and let me know your thoughts
> on
> > > this. Could I simply use HttpClient's default constructor without

> > > MultiThreading?
> > >
> > > Thanks a lot, in advance.
> > >
> > > Regards,
> > > Jaya
> > >
> >
> > Jaya
> >
> > Most likely you are seeing this exception because HttpClient spawns

> > a new thread per each connection attempt. In Java prior to version

> > 1.4 there is no way to control connection timeout using

> > java.net.Socket
> API.
> >
> > The only possibility to unblock a thread blocked in Socket#Socket

> > constructor is by using a controller thread.
> >
> > Basically you have two options:
> >
> > (1) Disable connection timeout by setting its value to 0. This will

> > prevent HttpClient from spawning a controller thread per connection

> > attempt
> >
> > (2) Upgrade to HttpClient 3.0, which can make use of Java 1.4

> > methods via reflection to control connection timeout when running in

> > a JRE >=
> > 1.4
> >
> > Hope this helps
> >
> > Oleg
> >
> >
> > > The logs are as follows.
> > > --------------------------------
> > > ####<29-Nov-05 11:57:54 GMT> <Error> <HTTP> <btnsc003-ukbg>

> > > <managed1_blwlp001> <ExecuteThread: '1' for queue: 'default'>
> <kernel
> > > identity>
> > > <> <101020>
> > >
> <[ServletContext(id=4834778,name=btportal_webapp,context-path=/btPorta
> > > l)
> > > ] Servlet failed with Exception>
> > > java.lang.OutOfMemoryError: unable to create new native thread
> > >         at java.lang.Thread.start(Native Method)
> > >         at
> > >
> org.apache.commons.httpclient.util.TimeoutController.execute(TimeoutCo
> > > nt
> > > roller.java:95)
> > >         at
> > >
> org.apache.commons.httpclient.util.TimeoutController.execute(TimeoutCo
> > > nt
> > > roller.java:116)
> > >         at
> > >
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:
> > > 67
> > > 5)
> > >         at
> > >
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpC
> > > on nectionAdapter.open(MultiThreadedHttpConnectionManager.ja
> > > va:959)
> > >         at
> > >
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> > > :6
> > > 61)
> > >         at
> > >
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java
> > > :5
> > > 29)
> > >         at
> > >
> com.bt.btcom.domain.dao.HttpClientDAO.executeMethod(HttpClientDAO.java
> > > :4
> > > 65)
> > >         at
> > >
> com.bt.btcom.domain.dao.HttpClientDAO.executeConnection(HttpClientDAO.
> > > ja
> > > va:308)
> > >         at
> > >
> com.bt.btcom.presentation.util.tsso.BVUrlConnectionHelper.openDAOConne
> > > ct
> > > ion(BVUrlConnectionHelper.java:114)
> > >         at
> > >
> com.bt.btcom.presentation.util.tsso.BVUrlConnectionHelper.getResponse(
> > > BV
> > > UrlConnectionHelper.java:52)
> > >         at
> > >
> com.bt.btcom.presentation.util.tsso.BVLoginHelper.login(BVLoginHelper.
> > > ja
> > > va:103)
> > >         at
> > >
> com.bt.btcom.presentation.util.tsso.ApplicationContext.processLogin(Ap
> > > pl
> > > icationContext.java:99)
> > >         at
> > >
> com.bt.btcom.presentation.util.tsso.SingleSignOn.login(SingleSignOn.ja
> > > va
> > > :92)
> > >         at
> > >
> com.bt.btcom.presentation.util.tsso.SSOFilter.doFilter(SSOFilter.java:
> > > 14
> > > 5)
> > >         at
> > >
> >
>
weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:
> > > 27)
> > >         at
> > >
> weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction
> > > .r
> > > un(WebAppServletContext.java:5523)
> > >         at
> > >
> weblogic.security.service.SecurityServiceManager.runAs(SecurityService
> > > Ma
> > > nager.java:685)
> > >         at
> > >
> weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppSer
> > > vl
> > > etContext.java:3156)
> > >         at
> > >
> >
>
weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.
> > > java:2506)
> > >         at
> > weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:234)
> > >         at

> > > weblogic.kernel.ExecuteThread.run(ExecuteThread.java:210)
> > > ####<29-Nov-05 11:57:54 GMT> <Error> <HTTP> <btnsc003-ukbg>

> > > <managed1_blwlp001> <ExecuteThread: '1' for queue: 'default'>
> <kernel
> > > identity>
> > > <> <101107>
> > >
> <[ServletContext(id=4834778,name=btportal_webapp,context-path=/btPorta
> > > l)
> > > ] Problem sending error page>
> > > javax.servlet.ServletException: unable to create new native thread
> > >         at
> > >
> weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatc
> > > he
> > > rImpl.java:327)
> > >         at
> > >
> weblogic.servlet.internal.ServletResponseImpl.sendError(ServletRespons
> > > eI
> > > mpl.java:517)
> > >         at
> > >
> weblogic.servlet.internal.ServletResponseImpl.sendError(ServletRespons
> > > eI
> > > mpl.java:370)
> > >         at
> > >
> weblogic.servlet.internal.WebAppServletContext.handleException(WebAppS
> > > er
> > > vletContext.java:3374)
> > >         at
> > >
> weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppSer
> > > vl
> > > etContext.java:3202)
> > >         at
> > >
> >
>
weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.
> > > java:2506)
> > >         at
> > weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:234)
> > >         at

> > > weblogic.kernel.ExecuteThread.run(ExecuteThread.java:210)
> > >
> > >
> > >
> > >
> > >
> > > *********************************************************
> > > Disclaimer:
> > >
> > > The contents of this E-mail (including the contents of the
> > enclosure(s) or attachment(s) if any) are privileged and

> > confidential material of MBT and should not be disclosed to, used by

> > or copied in
> any
> > manner by anyone other than the intended addressee(s).   In case you
> are
> > not the desired addressee, you should delete this message and/or

> > re-direct it to the sender.  The views expressed in this E-mail
> message
> > (including the enclosure(s) or attachment(s) if any) are those of

> > the individual sender, except where the sender expressly, and with

> > authority, states them to be the views of MBT.
> > >
> > > This e-mail message including attachment/(s), if any, is believed

> > > to be free of any virus.  However, it is the responsibility of the

> > > recipient to ensure that it is virus free and MBT is not

> > > responsible for any loss or damage arising in any way from its use
> > >
> > > *********************************************************
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> httpclient-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail:
> > > httpclient-user-help@jakarta.apache.org
> > >
> > >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail:

> > httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> httpclient-user-help@jakarta.apache.org
> >
> >
> > *********************************************************
> > Disclaimer:
> >
> > The contents of this E-mail (including the contents of the
> enclosure(s) or attachment(s) if any) are privileged and confidential

> material of MBT and should not be disclosed to, used by or copied in
any
> manner by anyone other than the intended addressee(s).   In case you
are
> not the desired addressee, you should delete this message and/or

> re-direct it to the sender.  The views expressed in this E-mail

> message (including the enclosure(s) or attachment(s) if any) are those

> of the individual sender, except where the sender expressly, and with

> authority, states them to be the views of MBT.
> >
> > This e-mail message including attachment/(s), if any, is believed to
> be free of any virus.  However, it is the responsibility of the

> recipient to ensure that it is virus free and MBT is not responsible

> for any loss or damage arising in any way from its use
> >
> > *********************************************************
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail:

> > httpclient-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> httpclient-user-help@jakarta.apache.org
> >
> >
>

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:

> httpclient-user-help@jakarta.apache.org
>

>

> *********************************************************
> Disclaimer:
>

> The contents of this E-mail (including the contents of the
enclosure(s) or attachment(s) if any) are privileged and confidential
material of MBT and should not be disclosed to, used by or copied in any
manner by anyone other than the intended addressee(s).   In case you are
not the desired addressee, you should delete this message and/or
re-direct it to the sender.  The views expressed in this E-mail message
(including the enclosure(s) or attachment(s) if any) are those of the
individual sender, except where the sender expressly, and with
authority, states them to be the views of MBT.
>

> This e-mail message including attachment/(s), if any, is believed to

> be free of any virus.  However, it is the responsibility of the

> recipient to ensure that it is virus free and MBT is not responsible

> for any loss or damage arising in any way from its use
>

> *********************************************************
>

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:

> httpclient-user-help@jakarta.apache.org
>

>


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


*********************************************************
Disclaimer:

The contents of this E-mail (including the contents of the enclosure(s) or attachment(s) if
any) are privileged and confidential material of MBT and should not be disclosed to, used
by or copied in any manner by anyone other than the intended addressee(s).   In case you are
not the desired addressee, you should delete this message and/or re-direct it to the sender.
 The views expressed in this E-mail message (including the enclosure(s) or attachment(s) if
any) are those of the individual sender, except where the sender expressly, and with authority,
states them to be the views of MBT.

This e-mail message including attachment/(s), if any, is believed to be free of any virus.
 However, it is the responsibility of the recipient to ensure that it is virus free and MBT
is not responsible for any loss or damage arising in any way from its use

*********************************************************

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message