lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Earwin Burrfoot (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2691) Consolidate Near Real Time and Reopen API semantics
Date Thu, 25 Nov 2010 17:00:18 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935794#action_12935794
] 

Earwin Burrfoot commented on LUCENE-2691:
-----------------------------------------

{quote}
bq. You're still okay with an API that allows you to reopen IRs on different directories?
Well, that's no good - we can catch this and throw an exc?
{quote}
I don't understand why should we bother with checking and throwing exceptions, when we can
prevent such things from compiling at all.
By using an API, that doesn't support reopening on anything different from original source.

bq. Really, there are two separate "things" open/reopen needs:
That's not true. Take a look at my WriterBackedReader above (or DirectoryReader in trunk).
It requires writer at least to call deleteUnusedFiles(), nrtIsCurrent().
So you can't easily reopen between Directory-backed and Writer-backed readers without much
switching and checking.

bq. r_ram.reload(); //Here we want to reload from the FSDirecotory?
Use MMapDirectory? It's only a bit slower for searches, while not raping your GC on big indexes.
Also check this out - https://gist.github.com/715617 , it is a RAMDirectory offspring that
wraps any other given directory and basically does what you want (if I guessed right).
It doesn't use blocking for files, so file size limit is 2Gb, but this can be easily fixed.
On the up side - it reads file into memory only after the size is known (unlike RAMDir), which
allows you to use huge precisely-sized blocks, lessening GC pressure.
I used it for a long time, but then my indexes grew, heaps followed, VM exploded and I switched
to MMapDirectory (with minor patches).

bq. What is missing is a "signal" from IR.reload() to RAMdirectory to slurp fresh information
from FSDirecory? 
There is zero need for any such signal. If a reader requests non-existing file from RAMDirectory,
it should check backing dir before throwing exception. If backing dir does have the file -
it is loaded and opened.
Why do you people love complicating things that much? :)

> Consolidate Near Real Time and Reopen API semantics
> ---------------------------------------------------
>
>                 Key: LUCENE-2691
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2691
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Grant Ingersoll
>            Assignee: Grant Ingersoll
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: LUCENE-2691.patch, LUCENE-2691.patch
>
>
> We should consolidate the IndexWriter.getReader and the IndexReader.reopen semantics,
since most people are already using the IR.reopen() method, we should simply add::
> {code}
> IR.reopen(IndexWriter)
> {code}
> Initially, it could just call the IW.getReader(), but it probably should switch to just
using package private methods for sharing the internals

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message