Return-Path: X-Original-To: apmail-httpd-users-archive@www.apache.org Delivered-To: apmail-httpd-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9744DDF69 for ; Tue, 31 Jul 2012 07:27:00 +0000 (UTC) Received: (qmail 71950 invoked by uid 500); 31 Jul 2012 07:26:57 -0000 Delivered-To: apmail-httpd-users-archive@httpd.apache.org Received: (qmail 71422 invoked by uid 500); 31 Jul 2012 07:26:57 -0000 Mailing-List: contact users-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: users@httpd.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list users@httpd.apache.org Received: (qmail 71356 invoked by uid 99); 31 Jul 2012 07:26:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Jul 2012 07:26:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of icicimov@gmail.com designates 209.85.161.173 as permitted sender) Received: from [209.85.161.173] (HELO mail-gg0-f173.google.com) (209.85.161.173) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Jul 2012 07:26:46 +0000 Received: by ggnp1 with SMTP id p1so6903497ggn.18 for ; Tue, 31 Jul 2012 00:26:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=z5G7EWFs9BkTZXKZTnCZ22oDe1CD1ywfNn5/IEMyHR0=; b=tco7Q+9KAqORx6d6Thba6mOrh+5eoT9Dh2daXlzZdOJcSnIJNtl8bC18/O1FdQSdVP P9qevGM57VSWLiAy4QHRWMp/4w42RmpzGw3SX2o6UBl8Tx5o+TvSx0d7jmLxvdaEPxoz BGXEllJigqntUiqYuEB7pCEdRbVpJWBVtE+rj0CK7BQb39LoqjhnvBFTdHRemGl2mX1m 7gYui4bMU0tMFxBCd141DMVMIktJdM4ONjlmt1Dqeflto8bjxxVzCU9O8R+YOzqwREzH 0b2KhPmHI50hAw/9K94YVaCwysFiuBD8iKonMDFviM7RE8JqYBQOcalmotdRLDkWHkLn sPWw== MIME-Version: 1.0 Received: by 10.60.29.230 with SMTP id n6mr21406249oeh.22.1343719585256; Tue, 31 Jul 2012 00:26:25 -0700 (PDT) Received: by 10.182.97.6 with HTTP; Tue, 31 Jul 2012 00:26:24 -0700 (PDT) In-Reply-To: References: Date: Tue, 31 Jul 2012 17:26:24 +1000 Message-ID: From: Igor Cicimov To: users@httpd.apache.org Content-Type: multipart/alternative; boundary=e89a8ff1c1468e93a604c61b1861 Subject: Re: [users@httpd] TimeOuts Received. --e89a8ff1c1468e93a604c61b1861 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jul 31, 2012 at 4:22 PM, gaurav shrivastava < g.shrivastava1985@gmail.com> wrote: > RHEL 5.5 > Apache version 2.2.1 > There are no connectivity issues due to firewall. > There is nothing which comes in the error logs. > > > > On Tue, Jul 31, 2012 at 6:27 AM, Igor Cicimov wrote: > >> On Mon, Jul 30, 2012 at 11:07 PM, gaurav shrivastava < >> g.shrivastava1985@gmail.com> wrote: >> >>> * >>> We automated our product >>> >>> Env - >>> >>> 4 Managed Serves - WLS weblogic (RHL) >>> 1 DB - Oracle (RHL) >>> 1 Proxy server - APACHE >>> >>> What i can see during the execution of and as the user load increases to >>> 180 + >>> I can see messages of timeouts at RPT console where response is not >>> received for the request. >>> Surprisingly i can not see any error logs for time outs any where in app >>> server logs or proxy server logs. >>> >>> >>> We did a test again with out proxy servers where requests were directly >>> hitting the app server every thing works well than no timeouts are noticted. >>> So it looks like the issue is with the proxy server do you have any >>> suggestions for any tunable parameters. >>> >>> >>> Some of the parameters currently used. >>> >>> >>> ThreadLimit 2000 >>> ServerLimit 10 >>> StartServers 6 >>> MaxClients 20000 >>> MinSpareThreads 2000 >>> #MinSpareThreads 100 >>> #MaxSpareThreads 20000 >>> MaxSpareThreads 15000 >>> ThreadsPerChild 2000 >>> MaxRequestsPerChild 0 >>> >>> >>> >>> Help will be highly appreciated. >>> * >>> >>> -- >>> *Cheers !! >>> >>> Gaurav ....* >>> >>> >> What OS is apache running on? What is apache version? What is your proxy >> configuration? What do you see in the log files, anything related to the >> timeouts from mod_proxy? Have you first tested the connectivity between >> apache and Weblogic meaning can you even connect from the apache box to the >> weblogic port on the other server? Any possible firewall between the >> servers? >> >> >> >> > > > -- > *Cheers !! > > Gaurav ....* > > Ok what else is between Apache proxy and those 4 x Weblogic servers? Are you using Apache as proxy load balancer or you have dedicated LB after it? How's your load balancing set up? Please post here your proxy configuration, including virtual host if the proxy is nested inside one, so we can try to find if anything wrong here. Your max number of concurrent client connections on your proxy is 20000. Do you have thread pool of 20000 configured in Weblogic as well, meaning 5000 threads per server? Also is that thread pool on each of the WLS shared with some other applications or not? My point is maybe the timeouts are really timeouts due to WLS (maybe one of them at time) is going out of resources. You said you tested the connection bypassing the proxy and it was fine. Was this at the time of the proxy timeouts? If not your test is irrelevant. --e89a8ff1c1468e93a604c61b1861 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, Jul 31, 2012 at 4:22 PM, gaurav shrivastava <<= a href=3D"mailto:g.shrivastava1985@gmail.com" target=3D"_blank">g.shrivasta= va1985@gmail.com> wrote:
RHEL 5.5=A0
Apache version 2.2.1
There are no connectivity is= sues due to firewall.
There is nothing which comes in the error l= ogs.


<= br>
On Tue, Jul 31, 2012 at 6:27 AM, Igor Cicimov <icicimov@gmail.com> wrote:
On Mon, Jul 30, 2012 at 11:07 PM, = gaurav shrivastava <g.shrivastava1985@gmail.com> w= rote:
We automated our product<= /div>

=
Env -

4 Managed Serves - WLS weblogic (RHL= )
1 DB - Oracle (RHL)
1 Proxy serv= er - APACHE=A0

What i ca= n see during the execution of and as the user load increases to 180 +
I can see messa= ges of timeouts at RPT console where response is not received for the reque= st.
Surprisingly i can not= see any error logs for time outs any where in app server logs or proxy ser= ver logs.=A0


We did a test = again with out proxy servers where requests were directly hitting the app s= erver every thing works well than no timeouts are noticted.
So it looks like the i= ssue is with the proxy server do you have any suggestions for any tunable p= arameters.


Some of the pa= rameters currently used.

<IfMod= ule worker.c>
ThreadLimit =A0 =A0 =A0 =A0 2000
ServerLimit =A0 =A0 =A0 =A0 10
StartServers =A0 =A0 = =A0 =A06
M= axClients =A0 =A0 =A0 =A0 20000
MinSpareThreads =A0 =A0 2000
#MinSpareThreads =A0 = =A0 100
#M= axSpareThreads =A0 =A0 20000
MaxSpareThreads =A0 =A0 15000
ThreadsPerChild =A0 = =A0 2000
M= axRequestsPerChild =A00
</IfModule>

=
Help will be highly appreciated.=

-- Cheers !!

Gaurav ....


What OS is apache run= ning on? What is apache version? What is your proxy configuration? What do = you see in the log files, anything related to the timeouts from mod_proxy? = Have you first tested the connectivity between apache and Weblogic meaning = can you even connect from the apache box to the weblogic port on the other = server? Any possible firewall between the servers?






--
Cheers !!

Gaurav ....

Ok what else is between Apache proxy and those 4 x Web= logic servers? Are you using Apache as proxy load balancer or you have dedi= cated LB after it? How's your load balancing set up?

Please post here your proxy configuration, including vi= rtual host if the proxy is nested inside one, so we can try to find if anyt= hing wrong here.

Your max number of concurrent cli= ent connections on your proxy is 20000. Do you have thread pool of 20000 co= nfigured in Weblogic as well, meaning 5000 threads per server? Also is that= thread pool on each of the WLS shared with some other applications or not?= My point is maybe the timeouts are really timeouts due to WLS (maybe one o= f them at time) is going out of resources.

You said you tested the connection bypassing the proxy = and it was fine. Was this at the time of the proxy timeouts? If not your te= st is=A0irrelevant.=A0
=A0=A0
--e89a8ff1c1468e93a604c61b1861--