clerezza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Spicar <daniel.spi...@gmail.com>
Subject Re: CRIS update
Date Thu, 25 Apr 2013 08:32:07 GMT
2013/4/25 Reto Bachmann-Gmür <reto@wymiwyg.com>

> Hi Daniel
>
> Thanks for investigating this.
>
>
> > - keep the optimize logic but use forceMerge method instead
> >
> > Ok, it says "This is a horribly costly operation, especially when you
> pass
> a small maxNumSegments; usually you should only call this if the index is
> static (will no longer be changed)."
>
> So I'm not sure what value to set to maxNum. What do you suggest?
>

Well optimize used to combine all segments into one, so maxNum should be 1
for the very same behavior. And yes, it used to be and still is horribly
inefficient. But it is also the only reliable way to free file descriptors
once open IndexWriters and IndexReaders have been optimized.


>
>
>
> > Subsequently I will look into CRIS and check if optimize can be replaced
> by
> > a merge policy or some other method.
> >
>
> Good. Having had a closer look into the current CRIS code I was wondering
> about the facet iimplementation why this doesn't seem to be mapped to the
> lucene.facet classes, i.e. it is countingthe facets manually rather than
> using org.apache.lucene.facet.search.CountFacetRequest, does this scale?
>

It's probably not going to scale to really large indices. When we
implemented it there was either no count facet request or something was
wrong with it. I will be changing this as the Lucene folks surely know much
better how to implement facets ;)


>
> Cheers,
> Reto
>
>
> >
> >
> >
> > 2013/4/19 Reto Bachmann-Gmür <reto@wymiwyg.com>
> >
> > > Ok good. I'm currently using the system with some data. If no problem
> > occur
> > > with the new version I'll remove those methods from the interfaces too.
> > >
> > > Cheers,
> > > Reto
> > >
> > >
> > > On Fri, Apr 19, 2013 at 2:39 PM, Daniel Spicar <dspicar@apache.org>
> > wrote:
> > >
> > > > Hi reto,
> > > >
> > > > No I did not check anything recently. I can review the changes if you
> > > like.
> > > > If optimize is not needed anymore that is good news. AFAIK it has
> been
> > > > mostly needed to keep the number of open file descriptors limited.
> > > >
> > > > Daniel
> > > >
> > > >
> > > > 2013/4/19 Reto Bachmann-Gmür <reto@apache.org>
> > > >
> > > > > Hi Daniel
> > > > >
> > > > > I'm wondering if you had a look at the recent changes to CRIS to
> work
> > > > with
> > > > > a more recent lucene version.
> > > > >
> > > > > With this update the optimize methods have become obsolete.
> > > > >
> > > > > Cheers,
> > > > > Reto
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message