Mailing-List: contact dev-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpComponents Project" <dev@hc.apache.org>
Received-SPF: Pass (protection.outlook.com: domain of PremierInc.com
 designates 216.221.240.180 as permitted sender)
 receiver=protection.outlook.com; client-ip=216.221.240.180;
 helo=C3PIMSHUB2.corp.premierinc.com;
From: "Idzerda, Edan" <Edan_Idzerda@PremierInc.com>
To: "dev@hc.apache.org" <dev@hc.apache.org>
Subject: How to tell if Async dispatcher thread is busy?
Thread-Topic: How to tell if Async dispatcher thread is busy?
Thread-Index: AQHSVLohjf0ncxjfb0Owex0M9howHA==
Date: Mon, 12 Dec 2016 21:15:23 +0000
Message-ID: <CY1PR07MB273265B8C1C2704989FD144C91980@CY1PR07MB2732.namprd07.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-Microsoft-Exchange-Diagnostics-untrusted: 1;CY1PR07MB2732;7:GqGmPeIrqc4uLniWOHDkl6Rbt2YQmFcK+nxnBV8MbUL3xG/ABh3qyxsv9aY7gXW2QiXxkDbhbLsXG0onfHek2b0o382wclNMrJw8WzHEUgmPs6TNGSMKrTajFkzHc8Y/NLbvsAID2VS5Cnz8SyNcLRKNtxLSu1HQ6m//RERDWL3y4/PcbcVoNEWMPsR0mOyWHj0n2C5Sb6bXrrJgG2v2BWCT1VvRX7saEq7Tx36S5J+U274ERiHYXz/yn1ot9GiGHgjmyB3kviyZLa3e31GiMfj3Q6DZDMeO40rJA4olqUypaaR+E+oXqchgaou5YRcVI3lds93odxcWIoi0I6V0Nhh+lTkocouWzSVTH77egXD5qJ+636624YInziwLHwMxDBR0cFKUk6gXW934X94I9a6M7yNfpzvBpZwm95zoJkgBKXwOdC2gLmUtDmXEhREZeU8PfiMAAFAMGQcYDbVAcQ==
X-Microsoft-Antispam-PRVS: 
	<BLUPR07MB324A8D16D82F6F1637D2E0191980@BLUPR07MB324.namprd07.prod.outlook.com>
x-exchange-antispam-report-test: 
	UriScan:(190756311086443);UriScan:(190756311086443);
x-exchange-antispam-report-cfa-test: 
	BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026)(6041248)(20161123555025)(20161123560025)(20161123562025)(20161123564025)(6072148);SRVR:CY1PR07MB2732;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2732;BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(13013025)(13020025)(13024025)(13023025)(3002001)(10201501046)(6055026)(6041248)(20161123562025)(20161123560025)(20161123564025)(20161123555025)(6072148);SRVR:BLUPR07MB324;BCL:0;PCL:0;RULEID:;SRVR:BLUPR07MB324;
x-forefront-prvs: 0154C61618
X-Forefront-Antispam-Report-Untrusted: SFV:NSPM;SFS:(10019020)(6009001)(7916002)(39450400003)(39840400002)(39410400002)(39850400002)(39860400002)(199003)(189002)(8676002)(106356001)(80792005)(106116001)(81156014)(2351001)(107886002)(68736007)(450100001)(97736004)(122556002)(3280700002)(3660700001)(102836003)(3846002)(9686002)(66066001)(92566002)(2900100001)(2501003)(38730400001)(6116002)(2906002)(86362001)(50986999)(7696004)(6916009)(77096006)(19627405001)(6506006)(6606003)(8936002)(110136003)(6436002)(33656002)(54356999)(5660300001)(7736002)(99286002)(105586002)(101416001)(81166006)(1730700003)(189998001)(76576001)(74316002)(85282002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR07MB2732;H:CY1PR07MB2732.namprd07.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
received-spf: None (protection.outlook.com: PremierInc.com does not designate
 permitted sender hosts)
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
Content-Type: multipart/alternative;
	boundary="_000_CY1PR07MB273265B8C1C2704989FD144C91980CY1PR07MB2732namp_"
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2732
X-IncomingHeaderCount: 35
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: BN3NAM01FT054.eop-nam01.prod.protection.outlook.com
X-MS-Office365-Filtering-HT: Tenant
X-Forefront-Antispam-Report: 
	CIP:216.221.240.180;IPV:CAL;SCL:-1;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10019020)(6009001)(7916002)(39410400002)(39860400002)(39840400002)(39850400002)(39450400003)(2980300002)(438002)(199003)(189002)(54356999)(50986999)(8676002)(2501003)(189998001)(512934002)(7696004)(110136003)(107886002)(99286002)(106116001)(6916009)(106466001)(356003)(2351001)(19627405001)(246002)(1730700003)(74316002)(33656002)(7736002)(7636002)(2900100001)(26826002)(8936002)(92566002)(76576001)(86362001)(9686002)(3846002)(450100001)(6116002)(84326002)(2906002)(102836003)(6506006)(66066001)(122556002)(3720700001)(38730400001)(80792005)(6070500001)(77096006)(61614004)(5660300001)(626004)(7099028)(85282002);DIR:OUT;SFP:1102;SCL:1;SRVR:BLUPR07MB324;H:C3PIMSHUB2.corp.premierinc.com;FPR:;SPF:Pass;PTR:mgat2.premierinc.com;A:1;MX:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: 1;BN3NAM01FT054;1:g6GbRWsG3DGKSDSXwopmetqk00Z1AKMtMktcebtczNgUv1BlTDjT26Kb3S/OPsTifr1RT4YrgwVVZ2Fdl7vXBMYxOVzM8KoxBa1YNxxeopa3ewqvU7a7CdR3uhOXlxo7ZszMXtVIAic7p0LvcJGf8k9wi7fdZAY4FVZQqArB0oA2RF03tqM/EfjqU16AWGKMmpOm1GSPXhVbgOY47jSvAt1/K9+UZhRnQbT1Sb8aGS8Xw9xbOaw+yaJMHZP21fQ2F2SqcmwL4feNz/knIq+k1Nea9iSqf5j9QnE2GI/RyI08y3y1ny0Kb3CmyFpEllrUYGj3JdcsoA14DeSd+Wa8wXgfj/488wSWFFWSKPYu+81OzB73ys12hvb7xQWF9baD7KswzwsbapjAHW4aBI2vtXvS2OFIggUTrU45MhaRB2ruEP9xMaFMnBgYVwKC6BjEkzMvg/XnJ5pBZo2LxddduPJHL5piPX69I/p6r9j8nyCCDKlH/6vTI4IfKxkITa4LqDQEtGYZdH767npK9DrQtnscfFO8tzKU3mHWGh11Jxq78G+SDVPF0Fjb1dLG6JHQrrbbZvcZh8mPkv6xu5jZ/g==
X-Microsoft-Antispam: 
	UriScan:;BCL:0;PCL:0;RULEID:(22001)(8251501002);SRVR:BLUPR07MB324;
X-Microsoft-Exchange-Diagnostics: 
	1;BLUPR07MB324;3:F8CXdy0g/vPmQ1P5TGLGxpWoPfvVhHL7HUh7t5U0vcUh380mIXNxMN1bvqxjLsWOiZAAI03jw1/DZ5ZSlVDpbNR+JMHdG+2b0hYeGVcowSr4LxpSiGMLP8WPdpQmBQnK0cxkmY4+Zx+n+bthtdudmwXIQuML/SLHBsVv89dexHnJgFUSVBgCiKHOpbWXkzGfHPYBwxefbYwUjFZx2OxvHJpMbwRHJRGmxZR66I8PzDFw2IKKy3g12rO2grSJc6Y8BKH/R6IIEuZ8Fqvg+EzIyZcZJbuYh/zMLlDGJmNeBvdDJgXL4cmqWdV/XLav0CO7mky0AD0o+6jsJEC74lcAIsVClnkpeDR546xbjaIg/0ZNdYDLYDbDDshi23gpJ618FnXSc5yyuJy0t+Pvlf5MAA==;25:mB/sBlNnHHtZMynXW0dOvS7mCML7NA5BCcLlihVEdtH/oJxC9Pp1GamARHxsKFdD5z5ilreI3T4re7XCCd2jjzJ6QrAMcLEh0+Y32MZgjp9eVCvBJVPg4Zr8uMpNRHFgFx26xk3vADSII0UpUDbJg+JK7yzCudaXUqb/+9EkNKF+MajYPgYJiYj+0oU0gghNNdv6cokKQvsVZUTUt0p3NHXWVQEVRMal/kq53x/EmA4jYhYMVI+8grUcBUjiYcAvqNn1GpVA8uH9O9TAl5OSmqtg26++P0jWZuPvSxdInI+hwBPVx4UFpKQizTbp8GbLjhfISIlPL+uekJXseDm9viJEBHzp/+Kvbex/vSGPOQ7R1erjviwbggD3pshHguuqWHMHUcyQWH440AVdmraN8Njik6mJyfpySHmrYnhqBonDpGV4hUrOhrqaj8wNEd4olxDgSt/jVUePkMxbF6lb/A==
X-Microsoft-Exchange-Diagnostics: 
	1;BLUPR07MB324;31:nxJIS8PG/MT0MG3mw8r4uff+49pj5HY+pgMS59iinsw27xvSZIPmf4BDlrKBflEWqIPfoQV+e9sPeeJRqLH/DE0vpON13UAwvmtslZst1AoQ2p17YU3GPvgb2wQgtnIW69HER3//8kb+9Nf/89ioA7c+xwrQLp1kn9Aawv2zWxi9Aypc+YSNMX9RJ1OKyaaLu9XHFLFroDyyTz2xadoc1OBtbLUe1l16nNE8hdjywnJVaioo0kNI82wPCe7iD4GRD+HUC0Gp8ajlI5lrwM9U6Q==;20:8DvZvQx/oychR9C17O4Mvf9C3LjMFrv2LPaKfcf+5xCIEfG2XBBFq1EratssFkkeYdfFicCKlTsiBFxKjnL41r0+CPhdDe3UzjS+3TcZlFVYHmQO9tbPDEB1EgfwDTOfgFRyxe1N7fcD7FBF+593ZgG3/vnywfncfrxXj0F8rG3nHvYc05mJIRwN5uiwHQ5Sq/Rz+Xk5E1y4R72iYB80VlkWeao/ftL/5bvS6yie2BGiuaYpHVW5xT3vods2APD1a2sIw3NcsVqozK9gvslqUJgBSZtDT84owQZDTBko8oBp/lDyQpKAfdtBDEfsuuliuT24wBWFzrS1yv6RyhstBwmd3/IUibILDafpXSSBPUpcOFfZyADjZXT6p/9ASDU3JBfuw+iqwJ+YZQ54y0N2tX0yy/zUNYt4x6yDsTIPC5P8Vka+SJbA3ecg79SOqdjJAQMqwFa8cFDeCrP7dCoANUwXn06OaH6yv6U21ekLxA9vI3tEFL0bh4mTONntvk/e
X-Microsoft-Exchange-Diagnostics: 
	1;BLUPR07MB324;4:3eCbnes3WXB5+0ekXLTIIZTuGwsT3TczURlRR0P3UtmmNbe6UVvHevZJhIGbPvJ2Q1YMuTmH3x+L/7ZCYPzmaK3uRTXHC2f5uexlOSdCej8GdRCncZ2gCDBPStFaviqyGgslloJLq7daOLqLQ4BUpBjslqduSR+j6YuHyMVVWg71HqhD8acJh05sLGdf5qy4sjp1+RCY0tYEp4mu9ZYQLmM7yPLddruWvUlKlewbmZi33VZiZsdgPWbQxIJahxVQc1GLSvQ0aOYT+GRJ7JSs9nMyQRkKAg9qmUBp1HiFRmF1hGwrLsNC5p6c/1/Ed5l83MP/itn8nziLfCBqM9jA8gDiV/3+MfJN81JwIlC6S66xOVtZfoqTbYnXlSvtdxBgZO3+bgqzz8AlvaKo53V9GmyFRz73ICEZu8mPX61S/XVaenKxlXxQz4Zg9qc3zRa6J8mdhM9p+L1WRpoLZMZXm95XLUYvRGNfVdlhcIsEI4ZAET6bi7hZE/HItv35RD96F3CFs8PNvTEE7QWtiQWtHiO43q4xymkusLq+G0YISTU0RTyn+JZJn1zt38HsCn+xCsCsrQ0KaFQii9SX1VdlkpBb5+v/4SLcWVaQSlOBZG6lL9Wztc5mwof843fSkB1PGJd6v7KUN6XL1lroAMu4BFQ3pfUZ31uaGsTFclefFWGbnrrDg4JivQyyWGP0ldHNtkE6Xw6mwWG6TwRiIJ1lTA==
X-Forefront-PRVS: 0154C61618
X-Microsoft-Exchange-Diagnostics: 
	=?us-ascii?Q?1;BLUPR07MB324;23:Sx2+4rp54wRumMOWH+Km9jryoMHyv7pxyUXbRcQyoQ?=
 =?us-ascii?Q?JqyQ8Gtqc5bCdDsa6OVE1AQ0W+iYBvtRFVg7XhSHaZpVS54xJ7Z+hsgrXatE?=
 =?us-ascii?Q?IXDn4mYhtze9nCzQR/jLvJYDXay7BWQQ7iMEEvLHKeUvxSQv0KQLErTBIYZ2?=
 =?us-ascii?Q?/7ZCEL45JaD/jX5pPQC8M06zXd+C1pT8ISk+QnG+CZ5VVCwN4tGco4M5A0+c?=
 =?us-ascii?Q?O8gLH0UnPd7CSUyPsrK7CxGwr4kO2GHoMZD6U3U2iYaU7U/ONcTqOFL5Ly5G?=
 =?us-ascii?Q?usK1OnUL8PNNeXzP22Sm+ftacjBthUDtI79grEAbABSJ9Kn8cT2t3x4Rh278?=
 =?us-ascii?Q?4tfw6KEgJpJKF8bkoHd+Wdy/R8F2+gbxIZcP3btE7s0/XvQD8EqIhqFX8Fw0?=
 =?us-ascii?Q?8qb+PhuuUGGjp9o95rPR2DxDS+WHnB1R6OH5JAnKgn6wu12KtRRhknturQg4?=
 =?us-ascii?Q?b8aclodr8jFft+RZyIcTb0WtL3QSX/yP0fG5OWYTo1rqUe/g4cMcU332lNG5?=
 =?us-ascii?Q?huFPkxqeRaoU4Bo8DeJTcvlSaBpYgvEBbw1KPWR1ull79BCl6YfoXL6OEsmz?=
 =?us-ascii?Q?7jnNK/RFuDSoYGqP9h5KqmpxRYWZuC+M3cDwxJerSL+wLUl2HMuulDvflPiN?=
 =?us-ascii?Q?bBjHYz+bwCBVDzEbvjZihV6YNQ04vcmVYasrhXNoGVDrG9cwPlx6wogr7ZWO?=
 =?us-ascii?Q?PgD/OXwuGBWLyQwjBi409W0NSranXDN/n5jn2rPH3JRMiTx2mjExJn9F4Owz?=
 =?us-ascii?Q?TPcdI4IS+zCs90jt8jWbhJRwq6TsCZSn2WPJW8cyzarK4FJUIUIZws+wXsXm?=
 =?us-ascii?Q?JKSnMYYrH4OXBEXDJU541z0VeB4lAqGEjBXCG4QDqdVZ4rV+aQzBmotoTplc?=
 =?us-ascii?Q?lDo8kl7iaoPoXiiWVTGmKCHCGPnG8mk5NuN5cC5U4OYNpKj6mYSUzD+T5XrO?=
 =?us-ascii?Q?FtPKyj9QsyasKml+VUWDW1hKyj7pgiw4357MJnriA7U9PxzQ4wgbvHLBSiz/?=
 =?us-ascii?Q?4YXN5EdjFUnvTqiDXXa2Ou1iaa5nwd/9b1Fg+Cm4yI5VqwwYP5QvKLYxi30A?=
 =?us-ascii?Q?ih6hFaZXwD5y0AI+yDC+XQy+Po5p0dFNJok4EGO3oLwMO1t1fLna/zqcKRj2?=
 =?us-ascii?Q?5uhw1ZKQI2YO3Y+rwWOiGo9FgNHNPnmm9FsdUwM2GFKqhDLHrEzmLSZqaZ5Z?=
 =?us-ascii?Q?MOFTWZZDs13Kusnu4mDMT5Coml5fXu/fpqkhXT7EDMutnFYaIrF/oGybqJZC?=
 =?us-ascii?Q?ss7wmy3+Tb4N7zSDi1KFD1L32xSFgWQ/olkTVuUOx/QHkJ1VlYWe7PqhoV+p?=
 =?us-ascii?Q?x/M146OF+DNGXNiZnyyzdXK406mvE/jVHReu0zUKQy3JIcn+eVI9/lP3TrZD?=
 =?us-ascii?Q?+kGJ8LmGD9TgPfa6jWwHIvdF4pl3V3rBNSJ2o7MQ4/xh2K?=
X-Microsoft-Exchange-Diagnostics: 
	1;BLUPR07MB324;6:DjvWVrLnFXF96/CREZnvdLnNORq/WBlV5vWcLLjW3J5SWZbRHveEcraAo/wAoSTzquTcGpbxy6bJ/aDC+fVeJ0dsZN5bTP4pNh0iQjmKKLaycs3ZAeO7dkHLPHhzfxsH76JOguEIzY2WO50/X2rMtmuKXsosa/FD8AIS0suyRQzN2HAITlRzzdI58fjkl5ZPhOj3E1C+eFTds4fHUAipdorWJBBLVQQB1SrD+AAEXXRbvxdg5CMpwxEnPpn7POrH1AhRVzxkZLVpOOwxu7J9+aWyuLB+dW1VFtB2jas8RHGCDGJkEx1lMIxCEtyObpHLCkEaalaCaCzc04Iw3AWW2TEpV6sIdrfDDPziD65sWBe2BjshaNd28ZbXr7XVfioCBAyIRGfcEsSmhHkS5qXuWtOncpF9gI1ldWy0VS7kWscO3mqv6jX2uf41BejzaeWrgyu0xtfQ9CesX/icy2WACw==;5:CNR60kqFufLGMHWCgzjgGGTCgGJ8fJckTl/sMauHzTMUMixCfkf65D8XQ7qAjbRfg0QNHhLORLvO6OXPtt2D1RIR5L+AHSyqqF7Lm2IGaeugIb0q10SiubQGaRy57pCUV4pk6Lel0IYAvBGGxZJ59aQ4/RRlKnz0wvbWOzL0z/4=;24:+Q0xIxTxLU3/BlH/Hpry7fwrBrtEBwct9fdAxo1gyOQIHJlmThqvS04TbKaEZ6tEWt9nMH6Keh0bF4NIsBmnuIqF90Kxe9GdbJFVxxopVrk=
X-Microsoft-Exchange-Diagnostics: 
	1;BLUPR07MB324;7:BINVw55Zurg+/SDEEnciPPMnwWC/8HKJRY1t+1Y/iAdHInfZ7UpnEQWtGv6k8ax+JtvfGR1Vetcza+epXmw5UdQPdYN4wEvPsAqtgm4/FqKCb+rQqINujQn2pFakaCySqebt90onHRj3+tXvuc1llASg15K208rvU0NiIib8XTxSwsawkPtA0oXFCWF6LWycD9zvZ6bU21Av/dTF1J6ZCg3iFq11fXOVAiWYuPvkbRKEARZld3xbIvw6DYNmg8wLQv9KK1jh0gXuwdoHCutKCjbked7eixxFsRqzPEjJIACG8kB3tPjIjPGx5HQOyio+iH398z8RJi2fjmLt2RHXuvvZ7k3KjkvzZJG12PK+eJr+vEoll8GffhBatzpB0sYg5NKWIITOT0+Jkck/7000TS2qK6G3cUgVYGCsslo9bJ8oZQ092/gbIibj7DEZgeeI9NxSL5IAh3SjLXu+Wln6sg==
X-OriginatorOrg: premierinc.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2016 21:15:45.9979
 (UTC)
X-MS-Exchange-CrossTenant-Id: b110eddf-23ae-457c-a6f3-734d592b2847
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=b110eddf-23ae-457c-a6f3-734d592b2847;Ip=[216.221.240.180];Helo=[C3PIMSHUB2.corp.premierinc.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR07MB324
archived-at: Mon, 12 Dec 2016 21:16:05 -0000

--_000_CY1PR07MB273265B8C1C2704989FD144C91980CY1PR07MB2732namp_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hello!  Our reverse proxy uses the Async Client pool to handle connections =
to backend servers.  We've been tracking a problem for a while where we obs=
erve the initial TCP connection is made, but no thread is available to hand=
le the SSL setup before a 10 second timeout expires.  We get into trouble b=
ecause some of our backend servers are very slow, and some of our clients d=
ownload very slowly.


I'm experimenting with a patch to AbstractMultiworkerIOReactor.addChannel()=
 to determine whether the next dispatcher thread is "busy."  My first try w=
as to look at bufferedSessions from the BaseIOReactor, and go through the l=
ist of dispatchers one time to see if I can find a free one.


        int i =3D Math.abs(this.currentWorker++ % this.workerCount);

        for (int j =3D 0; j < this.workerCount; j++) {

            if (this.dispatchers[i].getSessionCount() =3D=3D 0) {

                break;

            }

            i =3D Math.abs(this.currentWorker++ % this.workerCount);

        }

        this.dispatchers[i].addChannel(entry);

This seems to help us in MOST of the cases we see this issue in production,=
 but there still seem to be a small number of threads which collide.  I'm t=
esting a different version which looks at AbstractIOReactor "sessions" to d=
etermine thread busy state, but it never seems to show more than "1" sessio=
n if I look at the size after piling up slow connections on top of each oth=
er.

I have two questions:
    Is there a better way to determine whether a thread is busy?
    Would you be willing to accept a patch to make the dispatchers array in=
 AbstractMultiworkerIOReactor "protected" so I can implement my own Connect=
ingIOReactor that overrides addChannel() with my own thread selection model=
?

Thanks a lot for your help and for providing such a great library to the co=
mmunity!

- edan


--_000_CY1PR07MB273265B8C1C2704989FD144C91980CY1PR07MB2732namp_--