lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vsevel <v.se...@lombardodier.com>
Subject Re: Searching while optimizing
Date Tue, 24 Nov 2009 14:08:31 GMT

Hi, just to make sure I understand correctly... After an optimize, without
any reader, my index takes 30Gb on the disk. Are you saying that if I can
ensure there is only one reader at a time, it could take up to 120Gb on the
disk if searching while an optimize is going on? 

I did not get your 3X when there is no reader. In that situation isn't that
the nominal size?

different subject: I saw in 3.0.0RC1 that interrupting a merging thread was
being discussed. couldn't you do something similar for searches. I let my
users do full text searches on documents with over 50 fields. if using too
many wildcards, the search could take a long time. and rather than
restricting what they can do, I would rather let them cancel the search
gracefully. would that be something feasible?

Thanks,
vincent


Michael McCandless-2 wrote:
> 
> On Tue, Nov 24, 2009 at 1:44 AM, vsevel <v.sevel@lombardodier.com> wrote:
>>
>> 1) correct: I am using IndexWriter.getReader(). I guess I was assuming
>> that
>> was a privately owned object and I had no business dealing with its
>> lifecycle. the api would be clearer to rename the operation
>> createReader().
> 
> I just committed an addition to the javadocs that the caller is
> responsible for closing the returned reader.
> 
> I think createReader() isn't great either because it sound more
> expensive than it is -- under the hood, the returned reader is
> typically sharing many subreaders with the last reader obtained.  That
> sharing is what makes the reopen time fast.
> 
>> 2) how much transient disk space should I expect? isn't this pretty much
>> what the index writer javadoc said we should not do: "When running in
>> this
>> mode, be careful not to refresh your readers while optimize or segment
>> merges are taking place as this can tie up substantial disk space."
> 
> It is exactly what the javadoc says you should not do, but if you know
> the risks, go for it ;)
> 
> How much space is tied up depends on how often you reopen and how
> quickly you close the last reader.  If eg you aggressively close the
> last reader, such that effectively only one reader is open at once,
> then I think you're looking at worst case index consumes 4X it's
> "nominal" size (vs 3X if you don't open a single reader).
> 
> Mike
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/Searching-while-optimizing-tp26485138p26496505.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message