lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eks Dev (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2701) Expose IndexWriter.commit(Map<String,String> commitUserData) to solr
Date Mon, 07 Nov 2011 20:02:51 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145746#comment-13145746
] 

Eks Dev commented on SOLR-2701:
-------------------------------

@Erick, just go ahead and take it. 
I am not going to be working on this any time soon. At the moment I am using quck'n dirty
patched trunk version (moving target anyways) with extended CommitCommand to pass Map around
(sub-optimal approach? but does the work for now). 

Some thinking about it, maybe you find something useful: 
 
Take care, optimize and autoCommit do implicit commit (from user perspective, there is no
explicit transaction to commit where we could pass Map parameters). This, as a consequence,
requires Map<String, String> to be alive somewhere (DUH2 looks like the best place for
it). Of course, one needs to expose some user interfaces that will enable map mutation and
inquiry. This Map then becomes cached key-value pairs holder a user can change and solr offers
guaranties to commit it on implicit/explicit commit and read it on reload/rollback 

Rollback and restart, e.g. what happens to this map after restart (core reload)? I would suggest
populating it with committed values, on rollback as well.

As a summary: 
- One thing is low level mechanics, this is easy: all changes are local to DUH2, one Map<String,
String> and passing this instance to all commit commands you see there. Of course, reloading
it on index reload/rollback
- Much harder (at least for me): designing good user interface to maintain it, 
... explicit changes vie request handler (admin like operation)
... as parameter of the commit command (nice)
... somehow hooking into update chain elegantly (My primary use case! I keep track of the
max timestamp in this map (actually in AtomicLong, just populating Map on commit) to control
incremental updates, but my use case is dumb easy to support with patched CommitCommand as
I have only *explicit commits* (this wold not work with e.g. autoCommit, you would need Map
instance for it).   

e.g. Look at DIH, it uses internal counters and file system to persist it for this, that could
be much better served by lucene commit guaranties...


On another note, keeping real-time (not committed values) track of min/max values for user
defined fields would make sense for incremental update scenarios, I do not know if there is
something in lucene/solr for it already, but this is another, but somehow related issue...
 

Cheers, 
Eks 






                
> Expose IndexWriter.commit(Map<String,String> commitUserData) to solr 
> ---------------------------------------------------------------------
>
>                 Key: SOLR-2701
>                 URL: https://issues.apache.org/jira/browse/SOLR-2701
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>    Affects Versions: 4.0
>            Reporter: Eks Dev
>            Assignee: Erick Erickson
>            Priority: Minor
>              Labels: commit, update
>         Attachments: SOLR-2701.patch
>
>   Original Estimate: 8h
>  Remaining Estimate: 8h
>
> At the moment, there is no feature that enables associating user information to the commit
point.
>  
> Lucene supports this possibility and it should be exposed to solr as well, probably via
beforeCommit Listener (analogous to prepareCommit in Lucene).
> Most likely home for this Map to live is UpdateHandler.
> Example use case would be an atomic tracking of sequence numbers or timestamps for incremental
updates.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message