lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1313) Realtime Search
Date Tue, 05 May 2009 09:00:31 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705963#action_12705963
] 

Michael McCandless commented on LUCENE-1313:
--------------------------------------------


  * We shouldn't add FSD.setPrimaryExtensions?

  * The dual directories is continuing to push deeper (when I'm
    wanting to do the reverse).  EG, MergeScheduler.getDestinationDirs
    should not be needed?

  * We should no longer need IndexWriter.getFlushDirectory?  IE, IW
    once again has a single "Directory" as seen by IFD,
    DocFieldProcessorPerThread, etc.  In the NRT case, this is an FSD;
    in the non-NRT case it's the Dir that was passed in (unless, in a
    future issue, we explore using FSD, too, for better performance).

  * We can't up and change CMS.handleMergeException (breaks
    back-compat); can you deprecate old & add a new one that calls old
    one?  Let's have the new one take a Throwable and
    MergePolicy.OneMerge?

  * Instead of overriding "equals" (FSD.equals) can you change to
    "Directory.contains"?

  * IW's RAMDir usage still isn't factored in properly.  EG
    DW.doBalancRAM is not taking it into account.

  * Furthermore, we can't call writer.getRamDirSize() from
    DW.balanceRAM -- that's far too costly.  Instead, whenever RAMDir
    changes (deletes are applied, or a new RAM segment is created), we
    must push down to DW that usage with a new synchronized method.
    (I described this above).  We should remove
    IW.getRamDirSize()... ie, this size should always be "pushed on
    change", not "polled on read".  We change it rarely and read it
    incredibly often.

  * We don't need IW.getRamLogMergePolicy()?  Instead, let's ignore
    "MergePolicy.useCompoundFile()" when we are writing the new
    segment to RAMDir?  Likewise we should not cast RAMMergePolicy to
    LogMergePolicy in setRAMMergePolicy, nor turn off its CFS there.

  * I still don't think we need a separate RAMMergeScheduler; I think
    CMS should simply always run such merges (ie not block on max
    thread count).  IW.getNextMerge can then revert to its former
    self.

  * MergePolicy.OneMerge.segString no longer needs to take a
    Directory (because it now stores a Directory).

  * The mergeRAMSegmentsToDisk shouldn't be fully synchronized, eg
    when doWait is true it should release the lock while merges are
    taking place.


> Realtime Search
> ---------------
>
>                 Key: LUCENE-1313
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1313
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>    Affects Versions: 2.4.1
>            Reporter: Jason Rutherglen
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1313.jar, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch,
LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch,
LUCENE-1313.patch, lucene-1313.patch, lucene-1313.patch, lucene-1313.patch, lucene-1313.patch
>
>
> Realtime search with transactional semantics.  
> Possible future directions:
>   * Optimistic concurrency
>   * Replication
> Encoding each transaction into a set of bytes by writing to a RAMDirectory enables replication.
 It is difficult to replicate using other methods because while the document may easily be
serialized, the analyzer cannot.
> I think this issue can hold realtime benchmarks which include indexing and searching
concurrently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message