lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christoph Kaser <christoph.ka...@iconparc.de>
Subject Re: Document-Ids and Merges
Date Wed, 28 Mar 2012 07:37:51 GMT
Thank you for your answer!

That's too bad. I thought of using my own ID-field, but I wanted to save 
the additional indirection (from docId to my ID to my value).
Do document IDs remain constant for one IndexReader as long as it isn't 
reopened? If so, I could precalculate the indirection.

Best regards,
Christoph

Am 27.03.2012 18:15, schrieb Michael McCandless:
> In general how Lucene assigns docIDs is a volatile implementation
> detail: it's free to change from release to release.
>
> Eg, the default merge policy (TieredMergePolicy) merges out-of-order
> segments.  Another eg: at one point, IndexSearcher re-ordered the
> segments on init.  Another: because ConcurrentMergeScheduler runs
> different merges in different threads, they can finish in different of
> orders and thus alter how subsequent merges are selected.
>
> Really it's best if you assign your own (app-level) ID field and use
> that, if you need a stable ID.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Tue, Mar 27, 2012 at 3:29 AM, Christoph Kaser
> <lucene_list@iconparc.de>  wrote:
>> Hi all,
>>
>> I have a search application with 16 million documents that uses custom
>> scores per document using a ValueSource. These values are updated a lot (and
>> sometimes all at once), so I can't really write them into the index for
>> performance reasons. Instead, I simply have a huge array of float values in
>> memory and use the document ID as index in the array.
>> This works great as long as the index is not changed, but as soon as I have
>> a few new documents and deletions, index segments are merged (I suppose) and
>> the document IDs of existing documents change. Is there any way to be
>> informed when document IDs of existing documents change? If so, is there a
>> way to calculate the new document ID from the old one, so I can "convert" my
>> array to the new document IDs?
>>
>> Any help would be greatly appreciated!
>>
>> Best regards,
>> Christoph
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


-- 
Dipl.-Inf. Christoph Kaser

IconParc GmbH
Sophienstrasse 1
80333 München

www.iconparc.de

Tel +49 -89- 15 90 06 - 21
Fax +49 -89- 15 90 06 - 49

Geschäftsleitung: Dipl.-Ing. Roland Brückner, Dipl.-Inf. Sven Angerer. HRB
121830, Amtsgericht München




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message