lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Harris" <sharri...@gmail.com>
Subject Re: Re: Re: Re: Clustering IndexWriter?
Date Thu, 21 Sep 2006 20:27:32 GMT
Good question. May or may not be performant enough. Only time (and
testing) will tell. My guess is that it will depend heavily on the
rate in which the data changes (or read write ratio).

Believe me, I'm not proposing that everyone go out and cluster lucene
with terracotta dso. I'm really just playing, researching, learning.
I'm a firm believer in using the right tool for the right job and
would never claim that any product (especially one I wrote) is right
for everyone.

My guess is that some segment of the world cares a lot about realtime
coherent updates and some segment of the world needs blinding speed.
Part of my research is to gather the expertise of this group on these
issues.

Cheers,
Steve

On 9/21/06, Yonik Seeley <yonik@apache.org> wrote:
> While automatically clustering java objects sure sounds cool, I have
> to wonder what the performance ends up being.  Every small change to
> the clustered objects is broadcast to all the nodes, correct?
>
> Have you done any performance comparisons to see if this is a
> practical approach for Lucene?
>
> -Yonik
> http://incubator.apache.org/solr Solr, the open-source Lucene search server
>
>
> On 9/21/06, Steve Harris <sharrissf@gmail.com> wrote:
> > Fair question.
> >
> > All I did/need was take SegmentInfos and instead of subclassing Vector
> > I made it contain a Vector. Went from subclassing to aggregation. As
> > far as I could
> > tell from reading the code it would make no difference to anyone and
> > should have no performance impact (good or bad). It just allowed me to
> > cluster the IndexWriter with a RAMDirectory.
> >
> > Maybe a little background would help. Our clustering product doesn't
> > use java serialization and has no API. We just use a little config
> > where one points us to what you want clustered and what java
> > synchronization needs to be shared. One of the limitations that
> > currently exists is that we don't support clustering subclasses of
> > java collections.
> >
> > At this point I'm just experimenting to see if our product can cluster
> > lucene in a useful/performant way. When my experimenting is complete,
> > if everything is positive, I am going to write a blog on clustering
> > lucene indexes but it would be awkward to do that if the people who
> > run through the example have to change lucene code.
> >
> > Does this help?
> >
> > Cheers,
> > Steve
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message