lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanne Grinovero <sanne.grinov...@gmail.com>
Subject Re: Is there a way to share IndexReader data sensibly across independent callers?
Date Mon, 08 Feb 2016 15:10:08 GMT
Hi,
you should really try to reuse the same opened Directory, like you
suggest without closing it until your application is "done" with it in
all its threads (normally on application shutdown).
Keeping a Directory open will not lead to have open files, that is
probably caused by not closing the instances of IndexReader.

I'd highly recommend to use the ReaderManager for these reasons,
especially because handling these details across different threads
both correctly and efficiently can be tricky - I've learned that
myself when implementing similar things before the ReaderManager was
created.

HTH,
Sanne

On 5 February 2016 at 01:48, Trejkaz <trejkaz@trypticon.org> wrote:
> Hi all.
>
> Suppose 100 independent callers are opening the same index like this:
>
>     try (Directory directory = FSDirectory.open(path1);
>          IndexReader reader = DirectoryReader.open(directory))
>     {
>         // keeps reader open for a long time
>     }
>
> Someone complains that we're using a lot of memory, because each
> IndexReader has its own cache.
> We don't have any context that would tell us about other threads in
> the same JVM that might have opened the same reader, other than it
> ultimately going through our own code in both cases.
>
> Is there a proper way to share the underlying IndexReader data? I
> don't particularly care whether it's the same IndexReader but maybe it
> would help if it's at least the same SegmentReaders...
>
> Our existing attempts to use ReaderManager are not really working
> because there are issues with figuring out when it's okay to close a
> Directory. It seems like the answer is "never", because the caller who
> passes in the first one might still be using it. But if you never
> close it (which does seem to be the right way), Windows ruins your
> plans by refusing to let you delete the files.
>
> TX
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message