lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Rutherglen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1313) Realtime Search
Date Tue, 05 May 2009 19:54:30 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12706175#action_12706175
] 

Jason Rutherglen commented on LUCENE-1313:
------------------------------------------

{quote}RAMDir changes (deletes are applied, or a new RAM segment is
created), we must push down to DW that usage with a new synchronized
method.{quote}

Sounds like we create a subclass of RAMDirectory with this
functionality?

{quote}We don't need IW.getRamLogMergePolicy()?{quote}

Because we don't want the user customizing this?

{quote}We should no longer need IndexWriter.getFlushDirectory? IE, IW
once again has a single "Directory" as seen by IFD,
DocFieldProcessorPerThread, etc. In the NRT case, this is an FSD; in
the non-NRT case it's the Dir that was passed in (unless, in a future
issue, we explore using FSD, too, for better performance).{quote}

Pass in FSD in the constructor of DocumentsWriter (and others) as
before? 

{quote}I still don't think we need a separate RAMMergeScheduler; I
think CMS should simply always run such merges (ie not block on max
thread count). IW.getNextMerge can then revert to its former
self.{quote}

Where does the thread come from for this if we're using max threads?
If we allocate one, we're over limit and keeping it around. We'd need
a more advanced threadpool that elastically grows the thread pool and
kills threads that are unused over time. With Java 1.5 we can use
ThreadPoolExecutor. Is a dedicated thread pool something we want to
go to? Even then we can potentially still max out a given thread pool
with requests to merge one directory or the other. We'd probably
still need two separate thread pools. 

{quote}MergePolicy.OneMerge.segString no longer needs to take a
Directory (because it now stores a Directory).{quote}

Yeah, I noticed this, I'll change it. MergeSpecification.segString is
public and takes a directory that is not required. What to do? 

> Realtime Search
> ---------------
>
>                 Key: LUCENE-1313
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1313
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>    Affects Versions: 2.4.1
>            Reporter: Jason Rutherglen
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1313.jar, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch,
LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch,
LUCENE-1313.patch, lucene-1313.patch, lucene-1313.patch, lucene-1313.patch, lucene-1313.patch
>
>
> Realtime search with transactional semantics.  
> Possible future directions:
>   * Optimistic concurrency
>   * Replication
> Encoding each transaction into a set of bytes by writing to a RAMDirectory enables replication.
 It is difficult to replicate using other methods because while the document may easily be
serialized, the analyzer cannot.
> I think this issue can hold realtime benchmarks which include indexing and searching
concurrently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message