lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleg Kalnichevski (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-4509) Disable HttpClient stale check for performance and fewer spurious connection errors.
Date Tue, 02 Dec 2014 08:56:13 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231190#comment-14231190
] 

Oleg Kalnichevski commented on SOLR-4509:
-----------------------------------------

The idea behind connection check optimizations in HC 4.4 is rather straight-forward: given
the 'stale' check cannot be 100% reliable (the connection can turn 'stale' immediately after
passing the check but before a request is issued) there is no point checking the connection
indiscriminately before each and every request. It generally should be sufficient to check
those connections that are more likely to get 'stale'. Usually those are the connections that
have been sitting idle in the connection pool for a considerable period of time.

HC 4.4 is most likely going to get delayed by a month or two. So, there should enough time
to contribute an alternative solution if it turns out to be better than what we currently
have.

Oleg

> Disable HttpClient stale check for performance and fewer spurious connection errors.
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-4509
>                 URL: https://issues.apache.org/jira/browse/SOLR-4509
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>         Environment: 5 node SmartOS cluster (all nodes living in same global zone - i.e.
same physical machine)
>            Reporter: Ryan Zezeski
>            Assignee: Mark Miller
>            Priority: Minor
>             Fix For: 5.0, Trunk
>
>         Attachments: IsStaleTime.java, SOLR-4509-4_4_0.patch, SOLR-4509.patch, SOLR-4509.patch,
SOLR-4509.patch, SOLR-4509.patch, SOLR-4509.patch, baremetal-stale-nostale-med-latency.dat,
baremetal-stale-nostale-med-latency.svg, baremetal-stale-nostale-throughput.dat, baremetal-stale-nostale-throughput.svg
>
>
> By disabling the Apache HTTP Client stale check I've witnessed a 2-4x increase in throughput
and reduction of over 100ms.  This patch was made in the context of a project I'm leading,
called Yokozuna, which relies on distributed search.
> Here's the patch on Yokozuna: https://github.com/rzezeski/yokozuna/pull/26
> Here's a write-up I did on my findings: http://www.zinascii.com/2013/solr-distributed-search-and-the-stale-check.html
> I'm happy to answer any questions or make changes to the patch to make it acceptable.
> ReviewBoard: https://reviews.apache.org/r/28393/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message