lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Switching from Store.YES to Store.NO
Date Wed, 06 Jan 2010 12:01:44 GMT
This is a good question...

Merging will not alter the previously stored fields -- it will just
carry them forward to the next segment.

So any docs that have stored fields will retain them through merging.

(And, yes, merging is done "under the hood", separately from "indexing").

This is in contrast to other things.  EG, if you try to disable norms
on your fields, merging will sneakily turn norms back on, whenever a
segment containing "old" docs that have norms enabled is merged with
segments that had turned norms off.  You must fully re-index to
succeed in turning norms off.

Mike

On Wed, Jan 6, 2010 at 1:50 AM, Babak Farhang <farhang@gmail.com> wrote:
> I'm curious (:-)) about what do you mean by *adjusted*? Also, not sure
> I have the nomenclature here right, but isn't indexing functionally
> separate from merging segments? (You *index* to a segment which may,
> or may not, be later *merged* with other segments, no?)
>
> On Tue, Jan 5, 2010 at 11:28 PM, Chris Lu <chris.lu@gmail.com> wrote:
>> Just curious, will it be adjusted during indexing when merging segments?
>>
>> Thanks!
>>
>> --
>> Chris Lu
>> -------------------------
>> Instant Scalable Full-Text Search On Any Database/Application
>> site: http://www.dbsight.net
>> demo: http://search.dbsight.com
>> Lucene Database Search in 3 minutes:
>> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>> DBSight customer, a shopping comparison site, (anonymous per request) got
>> 2.6 Million Euro funding!
>>
>>
>> Michael McCandless wrote:
>>>
>>> Making that switch is fine.
>>>
>>> The change will not be retroactive, ie, all previously indexed docs
>>> with Store.YES will continue to store their fields.  But new docs
>>> won't store their fields if you specify Store.NO.
>>>
>>> I don't think this (what happens when certain schema changes happen
>>> mid-indexing) is well documented, in general.
>>>
>>> Mike
>>>
>>> On Tue, Jan 5, 2010 at 12:01 PM, Babak Farhang <farhang@gmail.com> wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> A review of the requirements of the project I'm working on has led us
>>>> to conclude that going forward we don't need Lucene to store certain
>>>> field values--just index.  Owing to the large size of the data, we
>>>> can't really afford to reindex everything, (Going forward, we plan to
>>>> treat these fields as if they were never stored.) Which brings up my
>>>> question.  What effect, if any, does switching to Store.NO have when
>>>> existing documents in the index already use Store.YES for that field?
>>>> Will the change take effect for the new docs added to the index?  Are
>>>> there any side-effects to consider?
>>>>
>>>> Also, more generally, since the Lucene API does not guard against
>>>> changing the document "schema" as the index is being built, I imagine
>>>> the effects of such mid-stream "schema" changes might be documented
>>>> somewhere.  Please share, if you know of such a document.
>>>>
>>>> Regards,
>>>> -Babak
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message