lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2193) Re-architect Update Handler
Date Mon, 30 May 2011 20:15:48 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041268#comment-13041268
] 

Mark Miller commented on SOLR-2193:
-----------------------------------

Hi Jason - long time no see :)

bq. I think the Solr ref counting code should go/exit, it's prone to pile up.

Can you elaborate? I'm def open to other approaches. I'm not yet sure why you are concerned
about pile up? Whenever a new SolrCore is opening, the previous ref will be closing. Have
you tried the patch and seen a problem here?

bq. Instead as with Twitter's system, a new reader is opened per query,
because the readers are lightweight enough.

Hmmm...opening a new IndexReader is not usually lightweight at all right now...which is why
we use reopen.

bq. If this patch isn't implementing NRT, what is the point?

Can you elaborate on why you don't think it's implementing NRT? I've tested basic indexing/searching
using wikipedia documents at about 50-100 documents a second, opening a new reader every second.
That felt pretty near-real-time to me, but the phrase is subjective. There are remaining issues
- Solr does not currently do everything at a per segment level as you know - but per segment
use cases will be very fast, and non per segment use cases will be *much* more NRT too.

You are questioning my whole patch :) Which I am happy about - but I'm wondering if you have
actually reviewed it - if not, I wish you would!

> Re-architect Update Handler
> ---------------------------
>
>                 Key: SOLR-2193
>                 URL: https://issues.apache.org/jira/browse/SOLR-2193
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>             Fix For: 4.0
>
>         Attachments: SOLR-2193.patch, SOLR-2193.patch, SOLR-2193.patch, SOLR-2193.patch
>
>
> The update handler needs an overhaul.
> A few goals I think we might want to look at:
> 1. Cleanup - drop DirectUpdateHandler(2) line - move to something like UpdateHandler,
DefaultUpdateHandler
> 2. Expose the SolrIndexWriter in the api or add the proper abstractions to get done what
we now do with special casing:
> if (directupdatehandler2)
>   success
>  else
>   failish
> 3. Stop closing the IndexWriter and start using commit (still lazy IW init though).
> 4. Drop iwAccess, iwCommit locks and sync mostly at the Lucene level.
> 5. Keep NRT support in mind.
> 6. Keep microsharding in mind (maintain logical index as multiple physical indexes)
> 7. Address the current issues we face because multiple original/'reloaded' cores can
have a different IndexWriter on the same index.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message