trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gang li <portl4t...@gmail.com>
Subject Re: Interim cache - High CPU usage
Date Mon, 12 Jan 2015 03:49:59 GMT
I don't think it is a good idea to use interim cache if the size of the
cache objects are very large. As the ssd disk is only 120GB,  the objects
on the HDD will be migrated to the ssd frequently, and the ssd storage is
meaningless as it will be overwritten quickly, and this will increase the
consumption of cpu and io.

Can you give more infos from the perf, such as call graph.


On Mon, Jan 12, 2015 at 4:52 AM, Daniel Biazus <daniel.biazus@azion.com>
wrote:

> Hi Guys,
>
> We' ve been using ATS as a reverse proxy, and a few week ago We started to
> use the interim cache feature in more intense way, caching objects with the
> average size of 200 MB and max size of 1 GB.
>
> *We have ~ 1 TB HDD as a default storage:*
>
> cat /etc/trafficserver/storage.config
>
> # ATS - Storage
> /dev/sda6 volume=1
>
> *And also, a 120 GB SDD storage as interim cache:*
>
> LOCAL proxy.config.cache.interim.storage STRING /dev/sdc1
>
> After 20 ~ 30 minutes in production with this configuration, We could
> notice a sudden CPU high usage, increasing up to 65 %, considering that our
> regular usage is 10 %. However the throughput still stable in 250 Mbps per
> box.
>
> We've found the following behavior using the perf top tool:
>
>    88.18%  traffic_server      [.]
> _Z15write_to_net_ioP10NetHandlerP18UnixNetVConnectionP7EThread
>      0.32%  traffic_server      [.] _ZN10NetHandler12mainNetEventEiP5Event
>      0.30%  [kernel]             [k] update_sd_lb_stats
>      0.29%  [e1000e]           [k] e1000e_check_ltr_demote
>      0.25%  [kernel]             [k] __ticket_spin_lock
>      0.24%  traffic_server      [.] _ZN7EThread13process_eventEP5Eventi
>      0.21%  [kernel]             [k] timerqueue_add
>      0.17%  libc-2.12.so       [.] epoll_wait
>      0.17%  libpcre.so.0.0.   [.] 0x00000000000100dd
>      0.14%  [kernel]             [k] __schedule
>
> 1) This behavior is easily reproduced caching* large objects with interim
> cache active*.
> 2) With interim cache *disabled*, this behavior* is not reproduced.*
>
>  As you can see, at the perf top output, the write_to_net_*io *function
> is responsible for this heavy CPU usage. We would like to hear of you guys,
> if anyone has faced a issue like that, or if you have any clues about this
> possible bug.
>
> Thanks & Regards,
>
> --
>
> Daniel Biazus
> infrastructure Engineering
> Azion Technologies
> Porto Alegre, Brasil +55 51 3012 3005 | +55 51 82279032
> Miami, USA +1 305 704 8816
>
> Quaisquer informações contidas neste e-mail e anexos podem ser
> confidenciais e privilegiadas, protegidas por sigilo legal. Qualquer forma
> de utilização deste documento depende de autorização do emissor, sujeito as
> penalidades cabíveis.
>
> Any information in this e-mail and attachments may be confidential and
> privileged, protected by legal confidentiality. The use of this document
> require authorization by the issuer, subject to penalties.
>
>

Mime
View raw message