lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <trej...@trypticon.org>
Subject Re: Is there a way to share IndexReader data sensibly across independent callers?
Date Tue, 09 Feb 2016 23:04:52 GMT
On Wed, Feb 10, 2016 at 3:17 AM, Michael McCandless
<lucene@mikemccandless.com> wrote:
> Why do you need to close the Directory?  It should be light weight.
> But if you really do need it, can't you subclass ReaderManager and
> override afterClose to close the directory?

I guess that's the next thing I'll try out. I am already subclassing
ReaderManager to wrap the readers after discovering that reader caches
don't work if I wrap from outside it.

As for the need to close it, it's true, in production we don't really
need to. But we happen to be using a Directory implementation that
checks that callers close things, and that check is only triggered
when we close the Directory at the moment. So it's just something that
adds diagnostics to help resolve other warnings about files not being
closed.

> So you essentially need to "lazy close" your ReaderManager, when there
> are no searches currently needing it?

That's more or less the right way to think about it. Actually each
session might make more than one search using the same reader and
reuse the same one for those, but when no sessions are running for
that index anymore I wanted to close it because Windows has annoying
file locking for read operations. If it weren't for Windows...

> Why not have a sync'd block, with a reference to the ReaderManager.
> Inside that block, if the reference is null, that means it's closed,
> and you open a new one.  Else, use the existing one.  Won't that do
> what you need w/o requiring full fledged reference counts?  Yes, it is
> a sync'd block around acquire/release, but I don't see how that can be
> avoided, and it'd be fast when the ReaderManager is already opened.

This is roughly what I currently have. At the moment I lock my entire
acquire() / release() methods, which maybe it's possible to reduce,
although I'm not entirely sure.

Concurrency is fiddly...

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message