lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Merge information in segment files
Date Fri, 16 Nov 2012 12:11:29 GMT
We do actually record this, in the segments "diagnostics" field ...
but that format is something that can suddenly "change" (ie it's not
an API w/ back compat).

Mike McCandless

http://blog.mikemccandless.com

On Fri, Nov 16, 2012 at 7:01 AM, Alan Woodward
<alan.woodward@romseysoftware.co.uk> wrote:
> Hi all,
>
> Is there any way of finding out if a segment is the result of a merge, or if it's just
new data?  I can't find anything in SegmentInfo that records this - if it isn't there, I'll
open a JIRA.
>
> Here's the use case:  I need to reload ExternalFileField data when segments are merged,
as the internal docids will all have changed, invalidating the EFF caches.  However, new segments
can just use default values (the EFF is used to store things like click rates, which are all
zero for new data).  At the moment, caches are refreshed after every commit.  But cache reloading
is heavy - if we can restrict it to only reload after a merge, then we save a lot of wasted
CPU and IO cycles.
>
> Thanks,
> Alan Woodward
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message