trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Kazlenka <pavel.kazle...@measurement-factory.com>
Subject Can ATS keep more than 500mbit/s traffic for single instance?
Date Thu, 21 Nov 2013 18:50:45 GMT
Hi gentlemen,

I'm trying to estimate maximum performance of ATS 4.0.2 on single server 
in forwarding proxy mode.
Server hardware is 6-core CPU ( +6 cores with HyperThreading), 12Gb RAM 
and two 10G NICs (one in client LAN and another in server LAN).
ATS is configured with hwloc support, caching is disabled, squid blob 
logging is disabled too.
I started from the config close to default: http://pastebin.com/AVQnJ4VL

But whatever I tried, I cannot force ATS to keep more than 500mbit/s (6k 
requests/s in my test) and do not start to drop requests. I tried to:
- limit working threads number to number of cores and let ATS decide how 
to map threads to cores;
- leave 6 cores to NIC interrupts and bind ATS working threads + accept 
thread to another cores so avoiding cores from changing tasks 
(ats/interrupts);
- play with memory related config variables: system.mmap_max, 
thread.default.stacksize, allocator.thread_freelist_size, etc.
- manually bind each ATS thread to dedicated CPU;
- combinations of the methods listed above;

As I stated, nothing helps. The only interesting result I got is when I 
left just one ATS working thread (no accept/disk threads) when this 
thread is assigned to CPU core manually and NIC interrupts are handled 
by another cores.  I got 6k r/s (500mbit/s) almost without adding 
latency on proxy side and up to 12k r/s (1gbit/s) with significant added 
latency (about 50-70 msec). Why ATS performs in 'single thread' mode 
better than in SMP mode? I don't know and hope you can clarify this.

I'm sure that the problem is not in OS tuning/hardware, because squid 
with similar config (6 workers, no caching) performs at 18k r/s 
(1.5gbit/s b/w).
Top utility also shows that there's lot of free resources: cpu cores are 
utilized by 30-35% each (20-25% of user + system and 5-8% of nic 
interrupts). There is much ram free (well, used for disk caches), NICs 
are not under heavy load too. I/O is not an issue too, because disabling 
logging at all doesn't give any performance boost.

So, does anyone have experience of configuring heavy loaded ATS on 
single server? Can you share your config/tricks in this case?

Are there any config options/tricks I should try?

And by any chance, if someone is interesting in ATS/squid performance 
comparison and wants to act as a consultant from ATS side, feel free to 
contact me by mail.

TIA,
Pavel


Mime
View raw message