lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Zezeski (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-4509) Disable Stale Check - Distributed Search (Performance)
Date Thu, 28 Feb 2013 15:23:13 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589601#comment-13589601
] 

Ryan Zezeski commented on SOLR-4509:
------------------------------------

Markus,

> Until now i've not seen real significant latency improvements when performing concurrent
stress tests. The response time still hovers between 70ms and 100ms with and without the patch
using today's trunk.

Fair enough, it could be that my particular setup had something to do with it.  I should verify
on a 2nd set of hardware.

What is this "concurrent stress test" you are running?  Is it something public I could run
as well?  How many shards is each query hitting and is each shard on its own physical machine?

Also, I'm going to attach a BTrace script.  If you could run this script it would be really
helpful as it would tell us how long the stale check is taking in your environment.  I included
instructions on running the script in the comments of the source.

> I'm not sure yet why these occur, maybe it's my Tomcat settings that do not match Jetty's
defaults.

My patch currently assumes a 50s idle timeout (or larger) on the server.  Any smaller and
you might see socket reset errors.

I'm not sure why you are seeing the errors you mentioned.  Your first trace doesn't match
up with the 4.1.3 HTTP Client code.  Has the client version been updated for the latest?



                
> Disable Stale Check - Distributed Search (Performance)
> ------------------------------------------------------
>
>                 Key: SOLR-4509
>                 URL: https://issues.apache.org/jira/browse/SOLR-4509
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>         Environment: 5 node SmartOS cluster (all nodes living in same global zone - i.e.
same physical machine)
>            Reporter: Ryan Zezeski
>            Priority: Minor
>         Attachments: IsStaleTime.java, SOLR-4509.patch
>
>
> By disabling the Apache HTTP Client stale check I've witnessed a 2-4x increase in throughput
and reduction of over 100ms.  This patch was made in the context of a project I'm leading,
called Yokozuna, which relies on distributed search.
> Here's the patch on Yokozuna: https://github.com/rzezeski/yokozuna/pull/26
> Here's a write-up I did on my findings: http://www.zinascii.com/2013/solr-distributed-search-and-the-stale-check.html
> I'm happy to answer any questions or make changes to the patch to make it acceptable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message