Return-Path: X-Original-To: apmail-trafficserver-users-archive@www.apache.org Delivered-To: apmail-trafficserver-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7D11D7642 for ; Fri, 30 Sep 2011 14:19:44 +0000 (UTC) Received: (qmail 67508 invoked by uid 500); 30 Sep 2011 14:19:44 -0000 Delivered-To: apmail-trafficserver-users-archive@trafficserver.apache.org Received: (qmail 67362 invoked by uid 500); 30 Sep 2011 14:19:43 -0000 Mailing-List: contact users-help@trafficserver.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@trafficserver.apache.org Delivered-To: mailing list users@trafficserver.apache.org Received: (qmail 67133 invoked by uid 99); 30 Sep 2011 14:19:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Sep 2011 14:19:43 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sridhar.basam@gmail.com designates 209.85.212.42 as permitted sender) Received: from [209.85.212.42] (HELO mail-vw0-f42.google.com) (209.85.212.42) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Sep 2011 14:19:37 +0000 Received: by vwl1 with SMTP id 1so1506694vwl.29 for ; Fri, 30 Sep 2011 07:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=o+GOjFCc4piQJEnvQXUtcFFdwkp6K0PNHdUW7DzV11E=; b=HjD7/NTx/yW2GKStS1s11+EYufCzzshYA/mcu04EXmlb938H1pxNVJeHuL/iWLBSRY e/0UEuKa+MBf5KKn9o4H6VKUeIBaPlmzmNBCeQ7qKV0JVhaXJbB0FGXcktyZ/x78+e+/ 4tci9L+7okPbxRM6jowkRcFlojswKOUvSORxQ= MIME-Version: 1.0 Received: by 10.52.34.171 with SMTP id a11mr10203542vdj.313.1317392355894; Fri, 30 Sep 2011 07:19:15 -0700 (PDT) Sender: sridhar.basam@gmail.com Received: by 10.220.108.137 with HTTP; Fri, 30 Sep 2011 07:19:15 -0700 (PDT) In-Reply-To: References: Date: Fri, 30 Sep 2011 10:19:15 -0400 X-Google-Sender-Auth: pQzkxQ4rrVP9ogi_parAgxoPCoU Message-ID: Subject: Re: High idle cpu utilization on RHEL6 From: sridhar basam To: users@trafficserver.apache.org Cc: dev@trafficserver.apache.org Content-Type: multipart/alternative; boundary=20cf307ca0b267259f04ae294f11 --20cf307ca0b267259f04ae294f11 Content-Type: text/plain; charset=ISO-8859-1 On Wed, Sep 28, 2011 at 3:35 PM, Brian Geffon wrote: > Hi, > I'm encountering something strange with ATS3.0.1 on Red Hat Enterprise > Linux 6, using a vanilla build with no modules enabled and the default > records.config /w zero entries in the remap file, ATS is idling at very high > CPU (around 15-20%). > > root 13509 0.0 0.0 58512 2392 ? Ss 19:15 0:00 > /usr/local/ats3.0.1plain//bin/traffic_cop > nobody 13511 0.1 0.0 480072 16632 ? Sl 19:15 0:00 > /usr/local/ats3.0.1plain/bin/traffic_manager > nobody 13521 ***16.9*** 0.1 1584628 114224 ? Sl 19:15 2:06 > /usr/local/ats3.0.1plain/bin/traffic_server -M -A,7:X > > So I used strace to try to determine what might be causing this, and here > is what i've found: > > [root@machine]# strace -c -p 13521 > Process 13521 attached - interrupt to quit > ^CProcess 13521 detached > % time seconds usecs/call calls errors syscall > ------ ----------- ----------- --------- --------- ---------------- > 100.00 3.589451 796 4510 epoll_wait > ------ ----------- ----------- --------- --------- ---------------- > 100.00 3.589451 4510 total > [root@machine]# > > > It appears that it's entirely epoll_wait, and each call is taking 796 > microseconds! So I have to concerns with this, first, why would epoll_wait > take such a long amount of time, 796 microseconds seems like a long time, > and more importantly, how could it possibly be called so frequently, does > ATS use a short timeout when doing epoll_waits? > > I would really appreciate any feedback regarding this, has anyone else > experienced this? Is there anywhere else I might look to determine the cause > of this? Could this be classified as _normal_ behavior? > > > It is working as designed. The epoll timeout is set to 0 or 10 msec on linux. Could there be very little or no load going through this instance? That would explain why you are only seeing epoll_wait. In that case there isn't really anything wrong, the process is just burning through epoll_waits looking for something to do when nothing is available. Once the process starts taking enough traffic, you will start to see user space and other kernel functions start to take cpu time. Sridhar --20cf307ca0b267259f04ae294f11 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Wed, Sep 28, 2011 at 3:35 PM, Brian G= effon <briang= effon@gmail.com> wrote:
Hi,
I'm encountering something strange with ATS3.0.1 on Red Hat Ente= rprise Linux 6, using a vanilla build with no modules enabled and the defau= lt records.config /w zero entries in the remap file, ATS is idling at very = high CPU (around 15-20%).

root =A0 =A0 13509 =A00.0 =A00.0 =A058512 =A02392 ? =A0 =A0 =A0 =A0Ss = =A0 19:15 =A0 0:00 /usr/local/ats3.0.1plain//bin/traffic_cop
nobody =A0 = 13511 =A00.1 =A00.0 480072 16632 ? =A0 =A0 =A0 =A0Sl =A0 19:15 =A0 0:00 /us= r/local/ats3.0.1plain/bin/traffic_manager
nobody =A0 13521 ***16.9*** =A00.1 1584628 114224 ? =A0 =A0 =A0Sl =A0 19:15= =A0 2:06 /usr/local/ats3.0.1plain/bin/traffic_server -M -A,7:X
So I used strace to try to determine what might be causing this= , and here is what i've found:

[root@machine]# strace -c -p 13521
Proce= ss 13521 attached - interrupt to quit
^CProcess 13521 detached
% time =A0 =A0 seconds =A0usecs/call =A0 =A0 calls =A0 =A0errors sy= scall
------ ----------- ----------- --------- --------- ----------------
100.00 =A0 =A03.589451 =A0 =A0 =A0 =A0 796 =A0 =A0 =A04510 =A0 =A0 = =A0 =A0 =A0 epoll_wait
------ ----------- ----------- --------- -= -------- ----------------
100.00 =A0 =A03.589451 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A04510 =A0 =A0= =A0 =A0 =A0 total
[root@machine]#=A0

<= div>
It appears that it's entirely epoll_wait, and each c= all is taking 796 microseconds! So I have to concerns with this, first, why= would epoll_wait take such a long amount of time, 796 microseconds seems l= ike a long time, and more importantly, how could it possibly be called so f= requently, does ATS use a short timeout when doing epoll_waits?

I would really=A0appreciate=A0any feedback regarding th= is, has anyone else experienced this? Is there anywhere else I might look t= o determine the cause of this? Could this be classified as _normal_=A0behav= ior?



It is workin= g as designed. The epoll timeout is set to 0 or 10 msec on linux. Could the= re be very little or no load going through this instance? That would explai= n why you are only seeing epoll_wait. In that case there isn't really a= nything wrong, the process is just burning through epoll_waits looking for = something to do when nothing is available. Once the process starts taking e= nough traffic, you will start to see user space and other kernel functions = start to take cpu time.

=A0Sridhar
--20cf307ca0b267259f04ae294f11--