lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Rochkind <rochk...@jhu.edu>
Subject Re: Using multiple CPUs for a single document base?
Date Tue, 31 May 2011 19:24:44 GMT
Yep, that could be it.

You certainly don't get _great_ concurrency support 'for free' in Java, 
concurrent programming is still tricky. Parts of Solr are surely better 
at it than others.

The one place I'd be shocked was just with multiple concurrent queries, 
if those weren't helped by multi-CPUs.   Multi CPUs won't neccesarily 
speed up any single query, they should just speed up the overall 
situation under heavy load.  Which has been my observation. And it may 
be that multi-CPU's don't speed up add/commit much, as you possibly have 
observed.

I do all my 'adds' to a seperate Solr index, and then replicate to a 
slave that actually serves queries. My 'master' that I do my adds to is 
actually on the very same server -- but I run it in an entirely 
different java container, in part to minimize any chance that it will 
end up competing for threads/CPUs with the slave serving queries, the OS 
level alone should ('should', famous last word)  balance it to a 
different cpu core. (Of course, there's still only so much total CPU 
avail on the machine).

On 5/31/2011 2:53 PM, Jack Repenning wrote:
> On May 31, 2011, at 11:29 AM, Jonathan Rochkind wrote:
>
>> I kind of think you should get multi-CPU use 'for free' as a Java app too.
> Ah, probably experimental error? If I apply a stress load consisting only of queries,
I get automatic multi-core use as expected. I could see where indexing new dox could tend
toward synchronization and uniprocessing. Perhaps my original test load was too add-centric,
does that make sense?
>
> -==-
> Jack Repenning
> Technologist
> Codesion Business Unit
> CollabNet, Inc.
> 8000 Marina Boulevard, Suite 600
> Brisbane, California 94005
> office: +1 650.228.2562
> twitter: http://twitter.com/jrep
>
>
>
>
>
>
>
>
>

Mime
View raw message