Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of
 ravikumar.govindarajan@gmail.com designates 209.85.214.48 as permitted
 sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAL8PwkZFpEm0MYGVV2ngkeOM4OQ6X+J63EVR=LXQOLM3FxYnTw@mail.gmail.com>
References: 
 <CAGW2whSLgXEpp-s7QNqcZteobN4Gc7Xsw71V66Qi944BYQ+Cqg@mail.gmail.com>
	<CAL8PwkY-zjW_UimaCXZaa9AZsEz6Ut1qDpXbK6vRKuvtn-8LJA@mail.gmail.com>
	<CAGW2whTnPwjJADDdt_dGeRJ3CoPv2EQVuZhMFeKdJ19LswXmkA@mail.gmail.com>
	<CAL8PwkZFpEm0MYGVV2ngkeOM4OQ6X+J63EVR=LXQOLM3FxYnTw@mail.gmail.com>
Date: Thu, 7 Nov 2013 22:48:09 +0530
Message-ID: 
 <CAGW2whRycZgS1M79rZsyLDk41M_rOAsr23N-HYoxg_Z2M=NHWA@mail.gmail.com>
Subject: Re: IndexReader close listeners and NRT
From: Ravikumar Govindarajan <ravikumar.govindarajan@gmail.com>
To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>
Content-Type: multipart/alternative; boundary=001a113337a427b19c04ea9973e2

--001a113337a427b19c04ea9973e2
Content-Type: text/plain; charset=ISO-8859-1

Thanks Mike.

If you look at my impl, I am using the getCoreCacheKey() only, but keyed
on a ReaderClosedListener and purging it onClose(). When NRT does reopens,
will it invoke the onClose() method for the expired-reader?. I saw that
FieldCacheImpl is using a CoreClosedListener, whereas I am using a
ReaderClosedListener. What is the difference between these two?

I will surely look at the FBS replacement you have pointed out. BTW, this
method actually runs during opening of an index for the first-time. May be
for clarity and organisation, I will refactor this code as you have
suggested

On Thursday, November 7, 2013, Michael McCandless wrote:

> Hi, a few comments on quickly looking at the code...
>
> It's sort of strange, inside the Weight.scorer() method, to go and
> build an IndexSearcher and run a whole new search, if the cache entry
> is missing.  Could you instead just do a top-level search, which then
> populates the cache per-segment?
>
> Also, FixedBitSet is better here than OpenBitSet: it should be a bit
> faster, because OpenBitSet is "elastic".
>
> When you use the core cache key, it does not change for a
> SegmentReader that was reopened with new deletes; this is the whole
> point of the core cache key. So, if deletes changed for a segment and
> you reopened, your cache entry will reuse whatever was created the
> first time for that segment.  If deletes matter, then it's usually
> best to look at liveDocs "live" and fold them in, instead of
> regenerating the whole cache entry.
>
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Thu, Nov 7, 2013 at 8:04 AM, Ravikumar Govindarajan
> <ravikumar.govindarajan@gmail.com <javascript:;>> wrote:
> > Thanks Mike. Can you help me out with one more question?
> >
> > I have a sample impl as below, where I am adding a ReaderClosedListener
> to
> > purge the BitSet.
> >
> > When using NRT with applyAllDeletes, old-reader will get closed and
> > new-reader will open. In such a case, will the below impl-cache also be
> > purged and re-built?
> >
> > I also saw that FieldCache uses a CoreClosedListener, instead of
> > ReaderClosedListener and I need such a functionality. It will be great to
> > maintain the BitSet cache at the cost of taking extra hit for testing
> > deletes.
> >
> > @Override
> >
> > public Scorer scorer(AtomicReaderContext context, boolean
> scoreDocsInOrder,
> > boolean topScorer, Bits acceptDocs) {
> >
> > Object key = context.getReader().getCoreCacheKey();
> >
> > OpenBitSet bitSet = cacheMap.get(key);
> >
> >     if (bitSet == null) {
> >
> >       reader.addReaderClosedListener(new ReaderClosedListener() {
> >
> >         @Override
> >
> >         public void onClose(IndexReader reader) {
> >
> >           Object key = reader.getCoreCacheKey();
> >
> >           cacheMap.remove(key);
> >
> >         }
> >
> >       });
> >
> >       final OpenBitSet bs = new OpenBitSet(reader.maxDoc());
> >
> >       //Add the empty bit-set first
> >
> >       cacheMap.put(key, bs);
> >
> >       IndexSearcher searcher = new IndexSearcher(reader);
> >
> >       //Do a search and populate the bitset
> >
> >       return bs;
> >
> >     }
> >
> >    //Proceed with scoring logic
> >
> > }
> >
> > --
> >
> > Ravi
> >
> >
> > On Thu, Nov 7, 2013 at 4:28 PM, Michael McCandless <
> > lucene@mikemccandless.com <javascript:;>> wrote:
> >
> >> You need to call .getCoreCacheKey() on each of the sub-readers
> >> (returned by IndexReader.leaves()), to play well with NRT.
> >>
> >> Typically you'd do so in a context that already sees each leaf, like a
> >> custom Filter or a Collector.
> >>
> >> Mike McCandless
> >>
> >> http://blog.mikemccandless.com
> >>
> >>
> >> On Thu, Nov 7, 2013 at 1:33 AM, Ravikumar Govindarajan
> >> <ravikumar.govindarajan@gmail.com <javascript:;>> wrote:
> >> > I am trying to cache a BitSet by attaching to
> >> IndexReader.addCloseListener,
> >> > using the getCoreCacheKey()
> >> >
> >> > But, I find that getCoreCacheKey() returns the IndexReader object
> itself
> >> as
> >> > the key.
> >> >
> >> > Whenever the IndexReader re-opens via NRT because of deletes, will it
> >> mean
> >> > that my cache will be purged, because a new IndexReader is opened?
> >> >
> >> > Are there ways to avoid this purging?
> >> >
> >> > --
> >> > Ravi
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org<javascript:;>
> >> For additional commands, e-mail: java-user-help@lucene.apache.org<javascript:;>
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org<javascript:;>
> For additional commands, e-mail: java-user-help@lucene.apache.org<javascript:;>
>
>

--001a113337a427b19c04ea9973e2--