hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Idzerda, Edan" <Edan_Idze...@PremierInc.com>
Subject How to tell if Async dispatcher thread is busy?
Date Mon, 12 Dec 2016 21:15:23 GMT
Hello!  Our reverse proxy uses the Async Client pool to handle connections to backend servers.
 We've been tracking a problem for a while where we observe the initial TCP connection is
made, but no thread is available to handle the SSL setup before a 10 second timeout expires.
 We get into trouble because some of our backend servers are very slow, and some of our clients
download very slowly.


I'm experimenting with a patch to AbstractMultiworkerIOReactor.addChannel() to determine whether
the next dispatcher thread is "busy."  My first try was to look at bufferedSessions from the
BaseIOReactor, and go through the list of dispatchers one time to see if I can find a free
one.


        int i = Math.abs(this.currentWorker++ % this.workerCount);

        for (int j = 0; j < this.workerCount; j++) {

            if (this.dispatchers[i].getSessionCount() == 0) {

                break;

            }

            i = Math.abs(this.currentWorker++ % this.workerCount);

        }

        this.dispatchers[i].addChannel(entry);

This seems to help us in MOST of the cases we see this issue in production, but there still
seem to be a small number of threads which collide.  I'm testing a different version which
looks at AbstractIOReactor "sessions" to determine thread busy state, but it never seems to
show more than "1" session if I look at the size after piling up slow connections on top of
each other.

I have two questions:
    Is there a better way to determine whether a thread is busy?
    Would you be willing to accept a patch to make the dispatchers array in AbstractMultiworkerIOReactor
"protected" so I can implement my own ConnectingIOReactor that overrides addChannel() with
my own thread selection model?

Thanks a lot for your help and for providing such a great library to the community!

- edan




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message