tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: Scripting mod_jk load-balancer member changes
Date Tue, 13 Jan 2015 14:46:25 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Rainer,

On 1/12/15 5:41 PM, Rainer Jung wrote:
> Am 12.01.2015 um 20:12 schrieb Christopher Schultz:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>> 
>> All,
>> 
>> On 1/12/15 2:01 PM, Christopher Schultz wrote:
>>> I'm running into a bit of difficulty scripting updates for 
>>> load-balancer members and thought I'd ask here before slogging 
>>> through the code to see if there are any requirements I'm
>>> missing.
>>> 
>>> Long story short, I'd like to script setting the "activation" 
>>> status for a particular load-balanced worker to something, like
>>> ACT or DIS.
>>> 
>>> I've written a python script to do this for me and I can
>>> confirm with Wireshark that it's sending what I expect. Here is
>>> the HTTP request that is being sent:
>>> 
>>> " POST /jk-status HTTP/1.1 Accept-Encoding: identity 
>>> Content-Length: 43 Host: localhost Content-Type: 
>>> application/x-www-form-urlencoded Connection: close
>>> User-Agent: mod_jk.py / Python-urllib
>>> 
>>> 
>>> from=list&cmd=update&sw=myworker&vwa=1&w=lb "
>>> 
>>> That's the whole request. The response I get is a 200
>>> response, but it's got the form content in it again, and not
>>> the "you will be redirected in 3 seconds" response that I get
>>> from the web UI.
>>> 
>>> I can think of a few things that I just wanted to sanity
>>> check:
>>> 
>>> 1. I need to provide all of the various values and not just
>>> the "vwa" value (which is the short-code request parameter
>>> value for the "activation" status).
>>> 
>>> 2. The status worker can't handle POST requests (the web
>>> interface uses GET).
>>> 
>>> 3. I've broken something else.
>>> 
>>> Can anyone confirm either 2 or 3 above, or both?
>> 
>> I tried switching to GET and so I'm using this URL, now:
>> 
>> GET /jk-status?from=list&cmd=update&sw=myworker&vwa=0&w=lb
>> HTTP/1.1 Accept-Encoding: identity Host: localhost Connection:
>> close User-Agent: mod_jk.py / Python-urllib
>> 
>> (Note that the load balancer worker's name is "lb" and the
>> balanced worker is called "myworker")
>> 
>> I'm getting a 200 response with the expected "You will be
>> redirected in 3 seconds" page, but the value for the activation
>> of this worker is not actually being updated.
>> 
>> So, it seems that the data must be in the URL (i.e. GET) but
>> something is still not working as I expect.
>> 
>> Can anyone shed some light on what I might be missing?
>> 
>> I can probably change the order of the parameters if that's
>> what's required... I'm using a python dictionary (an unordered
>> name/value hash) to build my request parameters which is
>> automatically converted to URL parameters, but they are
>> unordered. I could put them in order if that is likely to change
>> anything.
> 
> Only GET works currently.

I confirmed that the parameters must be in the URL, but it looks like
using a POST verb works just as well. I've changed my program to use
GET exclusively, since there's no reason to waste bytes with
Content-Length and Content-Type headers if I'm not going to send anything.

> It is easiest to do what you want in the browser GUI and then take
> the request URL including query string from the web server access
> log. And vice versa, does the URL do what you want, if you enter it
> in the browser?

That's what I started with. Well, I didn't bother to produce URLs from
the browser... I just looked at the <form>.

> At first sight, the query string looks good. Value "0" means
> "active". The "from=list" part should not be necessary. It will
> define, which redirect URL will be used in the response (in which
> you are not really interested).

I've removed "from=list"; I still get a <meta refresh> response which,
of course, I ignore.

> In case of problems, you can try the JkLogLevel debug for the
> status worker request. Status worker also logs quite verbosely what
> it does. Any changes done will be logged on info log level,
> problems as errors or warnings.

I can confirm that the status worker receives my new values, as per
the log file.

> Are you using 1.2.40 or an older version? Some older versions had 
> problems with updating data via status worker. It should be in the 
> changelog of the more recent version.

I'm using trunk, so 1.2.41-dev.

> Finally: make sure the status worker is not configured as
> read-only.

It's not. I can change the values from the web UI.

Something is definitely weird, here. What I had been doing was calling
the status worker to set the values and then immediately re-requesting
the status to see if the update had occurred. I was observing some
delay in being able to detect the change, so I introduced a 1/4s delay
between the update and status calls. That seemed to help. At first.

Now, what I'm observing is that if I do something like this:

$ mod_jk.py set activation=DIS (changes status)
$ mod_jk.py status (checks status)
  activation=ACT
$ mod_jk.py status
  activation=DIS
$ mod_jk.py status
  activation=DIS
$ mod_jk.py status
  activation=DIS
$ mod_jk.py status
  activation=ACT
$ mod_jk.py status
  activation=DIS
$ mod_jk.py status
  activation=ACT

Sometimes, I can get the web UI to bounce back and forth line this, too.

I added pid/tid to the access log file and I can see that the tid is
always the same, but that the pid is different. And the pid value
matches the activation value when I observe this odd behavior.

I think this is a problem of cross-process updates.

I'm not explicitly setting any shared-memory configuration. From
error_log:

[Mon Jan 12 12:18:07.410188 2015] [jk:warn] [pid 15054] No JkShmFile
defined in httpd.conf. Using default /usr/logs/jk-runtime-status

Guess what? The path /usr/logs/jk-runtime-status is invalid and does
not exist. From mod_jk.log:

[Tue Dec 30 12:17:37.159 2014] [34643:140735085486848] [debug]
jk_shm_calculate_
size::jk_shm.c (144): JK_SHM_SLOT_SIZE defined as 384, need at least 384
[Tue Dec 30 12:17:37.159 2014] [34643:140735085486848] [debug]
jk_shm_calculate_
size::jk_shm.c (178): shared memory will contain 1 ajp workers and 0
lb workers
with 0 members
[Tue Dec 30 12:17:37.160 2014] [34643:140735085486848] [error]
init_jk::mod_jk.c
 (3462): Initializing shm:/usr/logs/jk-runtime-status.34643 errno=2.
Load balancing workers will not function properly.

As soon as I went down the "pids disagree" path, I knew it had to be a
shared-memory problem. Glad I found it, and can configure around it.
I'll resume my testing with a sane environment, now ;)

Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJUtS/AAAoJEBzwKT+lPKRYTlwP+gMLN+TRH2TStrbrJa5MFEeV
47NL4+zC3yRiJO4nmKVMpK7PIafnW5m4byq92oKnyqQXUzzw4FMA/W/zk3C/N2Ww
3YeI1jouZhiLauw9N0DGHrx+vaPmedei2dKLgVcShEWzz2x5CPocEHSvsZQGQ5Hm
uc3kD++sZIufvxdLOpgKbfCKot3KC3q7J3ap+DASkCQWsPcmh2ePaYDhUX5KMJBj
xO9bGB6F6EoWokGXxdAZcK9T9XxdqRWY3MqcU/muj9DIwiDx9vW/L3NSsFGMZn1F
WP/Cu/6SS2vg5kFOise7Jx3M/aANNan+CxWV2qrvhECoWcX6oUh6w509cR06Uodh
edjT3FyHOYAGrL86ZA7siCuwwj9WV+yLMwbylcK4HIUHzD6+cG7+C0Ya1DAUmtHf
vBPPay4uzetyW1Fr1YvPvvFDXwJnvHoUjbf1H5J/4ZSW/uJ+r/9cJS0fCDoqZE+g
a9zNGr4AAyE5wQSH23SgBboh+ihfXZ+YDsaUjif9oWHFLo0dA676JGIoq2mdcd3d
6WHYVIyzu8sgXsWEG6mU36pZE2M8WF1Bype3+ntIYTV73hPJPMK5RLk7PQPhTQNA
fOUVUGIUy31bdLWQznjcqoV7ymNfXFY5AgbPxN0rAfgW9pPctwb3hR1zalHGQgZ9
MuRi07qgtzncKHLnlRRq
=8Nsf
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message