lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Lu" <chris...@gmail.com>
Subject Re: Clustering Lucene with 40 Servers
Date Wed, 27 Dec 2006 18:16:26 GMT
Simply using NAS as just another file directory will cause these
locks. You need to use your own logic to control when to re-open the
index reader.

I think you can look into Nutch's distributed file system to see
whether that can help.
-- 
Chris Lu
-------------------------
Instant Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com

On 12/27/06, Biggy <biggy97@web.de> wrote:
>
> Well try having say 30 servers try to write in the index at the same time and
> 10 others
> to read. You'll get enough locks to make a grown man cry. :)
>
>
> Scott Sellman wrote:
> >
> > Sorry if this seems naïve (I am new to Lucene), but why not keep one copy
> > of the Lucene index on a NAS and have it shared by all servers?
> >
> > -----Original Message-----
> > From: Biggy [mailto:biggy97@web.de]
> > Sent: Wednesday, December 27, 2006 7:57 AM
> > To: java-user@lucene.apache.org
> > Subject: Clustering Lucene with 40 Servers
> >
> >
> > I'm currently investigating the best ways of clustering Lucene.
> > I've heard of both Solr, Terracotta but do not know how well they scale.
> > Their examples talk of a 4 node cluster. This is way too small for my
> > needs.
> >
> > I have 30x JVMs each handling 3 requests/sec and each having their own
> > Lucene index. The index changes are propagated to the cluster members
> > using
> > JGroups messages. This solution has more than reached its limit as JGroups
> > has become unstable and a source of many JVMs crashes. Based on current
> > traffic trends I anticipate needing to upgrade to 40x + JVMS very soon.
> >
> > Can anybody suggest a way to effectivily cluster / replicate document
> > changes ?
> >
> > P.S.:
> > JMS is not a possible solution as this was the prior JGroups solution. We
> > had too many memory problems/queue full etc crashing the servers
> > thereafter.
> >
> > --
> > View this message in context:
> > http://www.nabble.com/Clustering-Lucene-with-40-Servers-tf2886546.html#a8064135
> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Clustering-Lucene-with-40-Servers-tf2886546.html#a8065033
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message