lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Fleming <afle...@hotmail.com>
Subject RE: Clustering Lucene with 40 Servers
Date Thu, 28 Dec 2006 07:33:37 GMT

Hello, 

I saw that Doug Cutting had an interesting solution for his Technorati website: 
http://www.mail-archive.com/lucene-user@jakarta.apache.org/msg12709.html

It sounds like it's a single-writer, many readers type of system, but quite robust and efficient.

Cheers, 

Adam 



----------------------------------------
> Date: Wed, 27 Dec 2006 10:46:47 -0800
> From: sharrissf@gmail.com
> To: java-user@lucene.apache.org
> Subject: Re: Clustering Lucene with 40 Servers
> 
> Some quick questions/points:
> 
> What is the update rate?
> 
> The number of nodes you described is no problem, the query rate would be
> no problem too (because they use read locks and act independently).
> 
> Do all nodes do updates or just 1? How often do these updates occur?
> 
> Probably best thing to do is give it a try and run a few tests.
> 
> Cheers,
> Steve
> 
> 
> On 12/27/06, Biggy <biggy97@web.de> wrote:
> >
> > I'm currently investigating the best ways of clustering Lucene.
> > I've heard of both Solr, Terracotta but do not know how well they scale.
> > Their examples talk of a 4 node cluster. This is way too small for my needs.
> >
> > I have 30x JVMs each handling 3 requests/sec and each having their own
> > Lucene index. The index changes are propagated to the cluster members using
> > JGroups messages. This solution has more than reached its limit as JGroups
> > has become unstable and a source of many JVMs crashes. Based on current
> > traffic trends I anticipate needing to upgrade to 40x + JVMS very soon.
> >
> > Can anybody suggest a way to effectivily cluster / replicate document
> > changes ?
> >
> > P.S.:
> > JMS is not a possible solution as this was the prior JGroups solution. We
> > had too many memory problems/queue full etc crashing the servers thereafter.
> >
> > --
> > View this message in context: http://www.nabble.com/Clustering-Lucene-with-40-Servers-tf2886546.html#a8064135
> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 

_________________________________________________________________
Fixing up the home? Live Search can help.
http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=wlmemailtaglinenov06
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message