lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1313) Realtime Search
Date Thu, 09 Apr 2009 09:20:12 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12697444#action_12697444
] 

Michael McCandless commented on LUCENE-1313:
--------------------------------------------

{quote}
The test we need to progress to is running the indexing side
endlessly while also reopening every X seconds, then
concurrently running searches
{quote}

Do you have a sense of what we'd need to add to contrib/benchmark to make this test possible?
 LUCENE-1516 takes the first baby step (adds a "NearRealtimeReaderTask").

{quote}
Usually there is a baseline QPS that is desired,
where the reopen delay may be increased to accommodate a lack of
QPS.
{quote}
Right -- that's the point I made on java-dev about the "freedom" we have wrt NRT's performance.

{quote}
The ram dir portion of the NRT indexing increases in speed when
more threads are allocated but those compete with search
threads, another issue to keep in mind.
{quote}
Well, single threaded indexing speed is also improved by using RAM dir.  Ie the use of RAM
dir is orthogonal to the app's use of threads for indexing?

{quote}
It might be good to add some default charting to
contrib/benchmark?
{quote}
I've switched to Google's visualization API (http://code.google.com/apis/visualization/) which
is a delight (they have a simple-to-use Python wrapper).  It'd be awesome to somehow get simple
charting folded into benchmark... maybe start w/ shear data export (as tab/comma delimited
line file), and then have a separate step that slurps that data in and makes a [Google vis]
chart.

> Realtime Search
> ---------------
>
>                 Key: LUCENE-1313
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1313
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>    Affects Versions: 2.4.1
>            Reporter: Jason Rutherglen
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1313.jar, LUCENE-1313.patch, LUCENE-1313.patch, lucene-1313.patch,
lucene-1313.patch, lucene-1313.patch, lucene-1313.patch
>
>
> Realtime search with transactional semantics.  
> Possible future directions:
>   * Optimistic concurrency
>   * Replication
> Encoding each transaction into a set of bytes by writing to a RAMDirectory enables replication.
 It is difficult to replicate using other methods because while the document may easily be
serialized, the analyzer cannot.
> I think this issue can hold realtime benchmarks which include indexing and searching
concurrently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message