lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Earwin Burrfoot (JIRA)" <>
Subject [jira] Commented: (LUCENE-1584) Callback for intercepting merging segments in IndexWriter
Date Tue, 07 Apr 2009 15:52:13 GMT


Earwin Burrfoot commented on LUCENE-1584:

bq. The problem is you need more information than simply "these segments got merged" to actually
do something interesting with your caches.
Okay, now I've thought a bit. What we need is a notification on which segments remained, which
are new and which got toasted, plus docid ranges for them. Their ancestry is irrelevant, because
you're right, to exploit it we also need deleted docs, and then replicate some of the merging
logic and it gets really messy from here. Dropping parts of the cache related to dead segments,
rebasing survivors and doing a fair-and-square load/uninversion/whatever for new ones is enough.

bq. Can you explain what's missing in Lucene's FieldCache?
It's not that easy to say. Our version was initially used only for sorting, but without concurrency
issues and with async warmup. But then we used it to load docs (way better than storing fields
and using IndexReader.document), tied up with our strongly-typed-fields code, added handling
for multi-valued fields, used it for faceted searches.
So now it is essentially just something different from Lucene field cache.

> Callback for intercepting merging segments in IndexWriter
> ---------------------------------------------------------
>                 Key: LUCENE-1584
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.4.1
>            Reporter: Jason Rutherglen
>            Priority: Minor
>             Fix For: 2.9
>         Attachments: LUCENE-1584.patch
>   Original Estimate: 96h
>  Remaining Estimate: 96h
> For things like merging field caches or bitsets, it's useful to
> know which segments were merged to create a new segment.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message