Return-Path: X-Original-To: apmail-trafficserver-users-archive@www.apache.org Delivered-To: apmail-trafficserver-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 45D3E9906 for ; Fri, 13 Apr 2012 14:04:09 +0000 (UTC) Received: (qmail 53754 invoked by uid 500); 13 Apr 2012 14:04:09 -0000 Delivered-To: apmail-trafficserver-users-archive@trafficserver.apache.org Received: (qmail 53724 invoked by uid 500); 13 Apr 2012 14:04:09 -0000 Mailing-List: contact users-help@trafficserver.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@trafficserver.apache.org Delivered-To: mailing list users@trafficserver.apache.org Received: (qmail 53714 invoked by uid 99); 13 Apr 2012 14:04:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Apr 2012 14:04:08 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.212.182] (HELO mail-wi0-f182.google.com) (209.85.212.182) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Apr 2012 14:04:01 +0000 Received: by wibhr14 with SMTP id hr14so2457269wib.5 for ; Fri, 13 Apr 2012 07:03:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:mime-version:content-type:subject:date:in-reply-to:to :references:message-id:x-mailer:x-gm-message-state; bh=C4DkKmQNMquDICbC2nLpMz8ws4FM35OwednVfdF9BYU=; b=YiHQapAbYwotvYrIOsdynw+9ldvmkgpsGITVNk8db+WEa7rCPLRD6kvl9AYp+/+alA OS7j0X7B9+RM50nMg2SMvEqA6q+RPAxN73/M6QulWAWVDmNYINqA1jOnSYcaynhXRnnS Mw9xM46KxPtTGLoZzNGmOUsVwJpK477J/66B+5aGjR/eFCxvIQ7bV2gVZ8iN+4xeHk5p l/i84ouJcRj07/1KzlcVZ0T9vecvr6aovWZfiD9QMUWQcnTTbcDdd8rLWBlKEEMNzf3U kLLytmDqHdGlk7Ma72sDFDxAA6sHg8T6s5+lSpS5AuuzstWg875GwBeAwKy2RJzOgM8K Utfg== Received: by 10.216.139.229 with SMTP id c79mr1272595wej.16.1334325819284; Fri, 13 Apr 2012 07:03:39 -0700 (PDT) Received: from [192.168.30.22] (80.174.93.243.dyn.user.ono.com. [80.174.93.243]) by mx.google.com with ESMTPS id gd4sm8298253wib.6.2012.04.13.07.03.37 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 13 Apr 2012 07:03:38 -0700 (PDT) From: =?iso-8859-1?Q?Jorge_Rom=E1n_Novalbos?= Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: multipart/alternative; boundary="Apple-Mail=_9293AFDE-5CAB-402B-A138-2F278438D946" Subject: Re: ATS concurrence Date: Fri, 13 Apr 2012 16:03:36 +0200 In-Reply-To: To: users@trafficserver.apache.org References: <1334067402.13612.18.camel@zym6400> <4F8443EC.606@cable.comcast.com> <5FA76BD7-3605-419B-9D4C-A88F32AA09D5@linux-it.es> <47F0DC7D-3191-47AF-9E37-F449ED6216EE@linux-it.es> <5BE3315B-80C1-4AC1-8A8A-7C7971C76F31@linux-it.es> Message-Id: X-Mailer: Apple Mail (2.1257) X-Gm-Message-State: ALoCoQlh5txKdljbe1Beg3lqOkbofbNp+3lvw9rYtZrwl9l5sqOXQm+KSkMNe1Xu5I5pdG4lT5Gf X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_9293AFDE-5CAB-402B-A138-2F278438D946 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 If i performance the test again with keep alive, i don't see any = TIME_WAIT connection but the timeouts still happen.=20 1) It was with default configuration 32768 61000, I have set this = param to 1024 65000. 2) No i always see this behavior. 3) When I perform the these with less concurrent request and keep alive = on, the report shows that almost all request were served by a keepalive = connection.=20 On 13/04/2012, at 11:24, Rohit Karlupia wrote: > 1) What do you get for cat /proc/sys/net/ipv4/ip_local_port_range ? =20= > 2) Does the TPS decreases over time? Or is it consistent for long = time?=20 > 3) Ab does report how many requests used keep alive connection. Check = if that value is significant. ab uses HTTP 1.0, not sure if ATS supports = keep-alive for http 1.0 connections.=20 >=20 > thanks! > rohitk >=20 > On Fri, Apr 13, 2012 at 1:53 PM, Jorge Rom=E1n Novalbos = wrote: > The ATS is not going back to the server, but i have a lot of TIME_WAIT = connections between ATS and "ab client machine", around 3600 TIME_WAIT = connection of 4000. >=20 > Any clue? >=20 >=20 > On 13/04/2012, at 09:28, Rohit Karlupia wrote: >=20 >> Can you verify that ATS is not going back to your servers?=20 >> Check netstat output on ATS machine?=20 >> - You should only have connections from client to ATS.=20 >> - Do you see connections in TIME_WAIT? How many?=20 >>=20 >> thanks! >> rohitk >>=20 >>=20 >> On Fri, Apr 13, 2012 at 12:47 PM, Jorge Rom=E1n Novalbos = wrote: >> Hi, thank for your answer!! >>=20 >> 1) I have tried with -k too, and i have timeouts too. >>=20 >> 2)I don't think that was the problem because I get the same timeout = from different clients from internet. anyway I have 10G internet = conexion. >>=20 >> 3) The ATS server load is very low during the test, never overcome 1. = The request never reach the origin servers because ATS is caching the = object, therefore the req per seconds should be much more high. >> 4) I have the same problem with statics objects. My origin servers = never could handle 10000 req/s, therefore I'm interesting in ATS. >> 5) Sure! >>=20 >> I don't know, but I think it's a problem (or setting) of the SO = tcp/ip stack. >>=20 >> Any other idea???=20 >>=20 >>=20 >> On 13/04/2012, at 08:34, Rohit Karlupia wrote: >>=20 >>> Multiple things can go wrong..;) >>>=20 >>> 1) You can use -k option with ab to keep the client connections = alive. That should be helpful. >>> 2) Divide the total bytes transferred by duration of the test to see = if you are close to bandwidth limits. (100Mbps or 1Gbps)=20 >>> 3) Are you maxing out on cpu on ATS or the application server = machine? ATS is a proxy. It will only add to latency. If the backend = server is exhausted, you probably need to put more servers and load = balance them using ATS to get your desired numbers. >>> 4) php is process based. What is behind ATS ? Apache? How many = processes?=20 >>> 5) Are you using caching on ATS?=20 >>>=20 >>> thanks! >>> rohitk >>>=20 >>> On Fri, Apr 13, 2012 at 1:48 AM, Jorge Rom=E1n Novalbos = wrote: >>> Hi everyone!! >>>=20 >>> I'm configuring a new ATS machine that it gonna server a lot request = per second (around 10000). >>>=20 >>> I have done several benchmark with ab from different machine to = achieve more concurrence but I can't get more than 3000 request per = second. >>>=20 >>> I have set several params in order to improve the performance: >>>=20 >>> echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle >>> echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse >>> echo "10" > /proc/sys/net/ipv4/tcp_fin_timeout >>>=20 >>> echo "10240" > /proc/sys/net/core/somaxconn >>>=20 >>> Besides, I have increased max open file allowed for nobody user to = 16384 (User that run ATS) >>>=20 >>> In the client side always get the same error : >>>=20 >>> apr_socket_recv: Connection timed out (110) >>>=20 >>> This is the ab command: >>>=20 >>> ab -c 600 -t 60 ''http://172.20.10.11/test.php" >>>=20 >>> I have increased the limit to 5000 in client side too. >>>=20 >>> Anybody knows how i can retrieve more concurrence in my server?????? >>>=20 >>> Thanks in advance!!! >>>=20 >>> Jorge >>>=20 >>=20 >>=20 >=20 >=20 --Apple-Mail=_9293AFDE-5CAB-402B-A138-2F278438D946 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 If i = performance the test again with keep alive, i don't see any TIME_WAIT = connection but the timeouts still happen. 

1) It = was with default configuration 32768 61000,  I have set this = param to 1024 65000.
2) No i always see this = behavior.
3) When I perform the these with less concurrent = request and keep alive on, the report shows that almost all request were = served by a keepalive = connection. 



On 13/04/2012, at 11:24, Rohit Karlupia wrote:

1) What do = you get for cat   /proc/sys/net/ipv4/ip_local_po= rt_range ?  
2) Does the TPS decreases over time? Or is = it consistent for long time? 
3) Ab does report how many = requests used keep alive connection. Check if that value is significant. = ab uses HTTP 1.0, not sure if ATS supports keep-alive for http 1.0 = connections. 

thanks!
rohitk

On Fri, Apr 13, 2012 at 1:53 PM, Jorge Rom=E1n = Novalbos <jroman@linux-it.es> = wrote:
The ATS is not going back to the server, = but i have a lot of TIME_WAIT connections between ATS and "ab client = machine", around 3600 TIME_WAIT connection of 4000.

Any clue?


On 13/04/2012, = at 09:28, Rohit Karlupia wrote:

Can = you verify that ATS is not going back to your servers?
Check netstat output on ATS machine?
 - You should only have = connections from client to ATS.
 - Do you see connections in = TIME_WAIT?  How many?

thanks!
rohitk


On Fri, Apr 13, = 2012 at 12:47 PM, Jorge Rom=E1n Novalbos <jroman@linux-it.es> wrote:
Hi,  thank for your = answer!!

1) I have tried with -k too, and i have = timeouts too.

2)I don't think that was the = problem because I get the same timeout from different clients from = internet. anyway I have 10G internet conexion.

3) The ATS server load is very low during the test, = never overcome 1. The request never reach the origin servers because ATS = is caching the object, therefore the req per seconds should be much more = high.
4) I have the same problem with statics objects. My origin servers = never could handle 10000 req/s, therefore I'm interesting in = ATS.
5) Sure!

I don't know, but I = think it's a problem (or setting) of the SO tcp/ip stack.

Any other = idea??? 


On = 13/04/2012, at 08:34, Rohit Karlupia wrote:

Multiple things can go wrong..;)

1) You can use -k option with ab to keep the client connections = alive. That should be helpful.
2) Divide the total = bytes transferred by duration of the test to see if you are close = to bandwidth limits. (100Mbps or 1Gbps) 
3) Are you maxing out on cpu on ATS or the application server = machine? ATS is a proxy. It will only add to latency. If the backend = server is exhausted, you probably need to put more servers and load = balance them using ATS to get your desired numbers.
4) php is process based. What is behind ATS ? Apache? How many = processes? 
5) Are you using caching on = ATS? 

thanks!
rohitk

On Fri, Apr 13, 2012 at 1:48 AM, Jorge Rom=E1n Novalbos <jroman@linux-it.es> = wrote:
Hi everyone!!

I'm configuring a new ATS machine that it gonna server a lot request per = second (around 10000).

I have done several benchmark with ab from different machine to achieve = more concurrence but I can't get more than 3000 request per second.

I have set several params in order to improve the performance:

echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle
echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse
echo "10" > /proc/sys/net/ipv4/tcp_fin_timeout

echo "10240" > /proc/sys/net/core/somaxconn

Besides, I have increased max open file allowed for nobody user to 16384 = (User that run ATS)

In the client side always get the same error :

apr_socket_recv: Connection timed out (110)

This is the ab command:

ab -c 600 -t 60 ''http://172.20.10.11/test.php"

I have increased the limit to 5000 in client side too.

Anybody knows how i can retrieve more concurrence in my server??????

Thanks in advance!!!

Jorge



=



= --Apple-Mail=_9293AFDE-5CAB-402B-A138-2F278438D946--