trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vaillancourt, Tim" <>
Subject Multiple Trafficserver Instances on one system
Date Tue, 03 Apr 2012 23:48:09 GMT
Hello Traffiserver users!

I'd like to start off by thanking the ASF, Yahoo!/Inktomai and other contributes for making
(and donating) this great piece of software! My team is currently implementing Trafficserver
in hopes to replace many Apache HTTPd/mod_proxy proxy servers in our team's production.

I am e-mailing the list due to an issue (or PEBKAC) I am experiencing when running several
instances of Trafficserver on one system, under custom directories.

In our environment we install what we call "Binary packages" and "Instance packages", meaning
we will install a compiled "Binary package" version of Trafficserver to say '/opt/tools/trafficserver3'
and use that set of binaries in several smaller "Instance packages" under say '/opt/instances/ts_uploadprx'.
Upon install of an "Instance Package" we symlink the bin, lib, modules dir over from the "Binary
package" ('/opt/tools/trafficserver3') into our instance package, but we'll have a unique
'conf' dir for the given instance. This is working quite well for us with standard Apache
HTTPd, allowing us to spawn several unique instances of Apache, all using one set of binaries.

When we try this methodology with Trafficserver 3.0.4, we run into several issues preventing
Trafficserver to function.

Here is an example of an ls on our "ts_uploadprx" instance (notice binaries are symlinked
while cache, conf, logs and share are instance-specific):

[tim@app013 ts_uploadprx]$ ls -al
total 44
drwxr-xr-x  6 root root  4096 Apr  3 00:08 .
drwxr-xr-x 11 traf traf  4096 Apr  2 23:48 ..
lrwxrwxrwx  1 root root    34 Apr  2 23:48 bin -> /opt/tools/trafficserver3/bin
drwxrwxr-x  2 root traf  4096 Apr  2 23:47 cache
drwxrwxr-x  3 root traf  4096 Apr  2 23:48 conf
lrwxrwxrwx  1 root root    38 Apr  2 23:48 include -> /opt/tools/trafficserver3/include
lrwxrwxrwx  1 root root    34 Apr  2 23:48 lib -> /opt/tools/trafficserver3/lib
drwxrwxr-x  2 root traf  4096 Apr  3 00:14 logs
lrwxrwxrwx  1 root root    38 Apr  2 23:48 modules -> /opt/tools/trafficserver3/modules
drwxrwxr-x  2 root traf  4096 Apr  2 23:47 share

To allow this all to function, I had to get trafficserver to source different values for TS_DAEMON,
TC_DAEMON, TM_DAEMON, etc on startup so I created a 'ts-defaults' script (similar something
you'd put into /etc/defaults/) to set the environment variables for my instance (here is me
sourcing 'ts-defaults' from cmd-line):

[tim@app013 ts_uploadprx]$ set -x; . ts-defaults; set +x
+ . ts-defaults
++ MY_PATH=/opt/instances/ts_uploadprx
++ TC_NAME=traffic_cop
++ TM_NAME=traffic_manager
++ TS_NAME=traffic_server
++ TC_DAEMON=/opt/instances/ts_uploadprx/bin/traffic_cop
++ TM_DAEMON=/opt/instances/ts_uploadprx/bin/traffic_manager
++ TM_DAEMON_ARGS='-path /opt/instances/ts_uploadprx/conf -recordsConf /opt/instances/ts_uploadprx/conf/records.config'
++ TS_DAEMON=/opt/instances/ts_uploadprx/bin/traffic_server
++ TS_DAEMON_ARGS='-d /opt/instances/ts_uploadprx'
++ TC_PIDFILE=/opt/instances/ts_uploadprx/logs/cop.lock
++ TM_PIDFILE=/opt/instances/ts_uploadprx/logs/manager.lock
++ TS_PIDFILE=/opt/instances/ts_uploadprx/logs/server.lock
++ STDOUTLOG=/opt/instances/ts_uploadprx/logs/traffic_server.stdout
++ STDERRLOG=/opt/instances/ts_uploadprx/logs/traffic_server.stderr
+ set +x

And I modified 'bin/trafficserver' to load this script if it exists (after the other lines
that source defaults files):

[tim@app013 ts_uploadprx]$ cat -n bin/trafficserver|grep -P "\s+143"
   143  test -f /opt/instances/ts_uploadprx/ts-defaults && . /opt/instances/ts_uploadprx/ts-default

Upon starting trafficserver up, I can see the 'bin/trafficserver' script get my new environment
variables from my 'ts-defaults' script and everything starts, but I notice only 'traffic_cop'
runs under the correct instance dir and settings and that 'traffic_manager' and 'traffic_server'
run out of the '/opt/tools/trafficserver3' "Binary package" path, causing logs, configs, etc
to be loaded from the "Binary Package" path and not my "Instance Package" path and in general
nothing works, and the unique settings are not loaded:

[tim@app013 ts_uploadprx]$ ps aux|grep traffic
tim       1028  0.0  0.0  61184   748 pts/0    S+   00:31   0:00 grep traffic
root     32618  0.0  0.0  61476  2600 ?        Ss   00:24   0:00 /opt/instances/ts_uploadprx/bin/traffic_cop
traf     32620  0.0  0.3 221348 15140 ?        Sl   00:24   0:00 /opt/tools/trafficserver3/bin/traffic_manager
traf     32630  0.0  1.5 182356 61424 ?        Sl   00:25   0:00 /opt/tools/trafficserver3/bin/traffic_server
-M -A,7:X

As you can see in my 'ts-defaults' script, I try passing "-path <Instance Path>" to
'traffic_manager' and "-d <Instance Path>" to 'traffic_server', but this doesn't cause
it to load under my "Instance package" path.

1) Have I configured something wrong in my setup?
2) Is what I am trying to achieve not possible in Trafficserver 3.0.4?
3) Are the paths that 'traffic_manager' and 'traffic_server' run under compiled in, and thus
are not overridable?

Any help or push in the right direction is most appreciated. Our team cannot wait to start
using this project for real and are just blocked on this likely minor issue. For completeness,
here is the './configure' line we're using for 3.0.4 in our SPEC file for our RPM:

CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS ;
CXXFLAGS="${CXXFLAGS:-%optflags}" ; export CXXFLAGS ;
FFLAGS="${FFLAGS:-%optflags}" ; export FFLAGS ;

./configure --host=%{_host} --build=%{_build} \
    --target=%{_target_platform} \
                --prefix=%{prefix}/trafficserver3-%{version} \
                --with-user=%{run_user} \
                --with-group=%{run_group} \
                --enable-layout=Apache \
                --disable-debug \
                --disable-fast-sdk \
                --disable-purify \
                --disable-libev \
                --disable-micro \
                --with-openssl \

Thank you so much for your time!

Tim Vaillancourt
Electronic Arts -
View raw message