lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-65) autoCommit/autoOptimize implementation + multithreaded document adding
Date Fri, 03 Nov 2006 17:01:17 GMT
    [ http://issues.apache.org/jira/browse/SOLR-65?page=comments#action_12447010 ] 
            
Yonik Seeley commented on SOLR-65:
----------------------------------

The multithreaded stuff looks very nice!

I like the idea of separating out the autoCommit functionallity into a tracker.

I notice that in closeWriter() you call tracker.didCommit()... but I think of a commit more
as making sure everything is on stable storage, and making those changes visible.  closeWriter()
is also currently called as an implementation side effect of delete-by-query.

There are some different uses for commit control that I can think of:
 1) users don't want to manually send commits, and want to ensure that any changes made to
the index will be visible within a certain amount of time.
 2) in the event of multiple updaters all calling commit often, the index owner wants to limit
the commit frequency to avoid search degredation (or OOM errors from too many overlapping
on-deck searchers).
 3) there are infrequent single updates to an index, but when they happen a commit should
be done ASAP to lower the latency till visibility.  Subject to limitations in (2)

It's possible people might even want tuning on the fly.
For instance, if you are about to do a compete index rebuild, it would be nice to be able
to turn off any autocommitting or commits alltogether to avoid partial indexes being visible.
 This could be done with special update commands, or perhaps flags on the <add> command?


> autoCommit/autoOptimize implementation + multithreaded document adding
> ----------------------------------------------------------------------
>
>                 Key: SOLR-65
>                 URL: http://issues.apache.org/jira/browse/SOLR-65
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>            Reporter: Mike Klaas
>         Assigned To: Mike Klaas
>         Attachments: autocommit_patch.diff
>
>
> Basic implementation of autoCommit/autoOptimize functionality, plus overhaul of DUH2
threading to reduce contention

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message