lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Per Steffensen (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (SOLR-3178) Versioning - optimistic locking
Date Thu, 19 Apr 2012 18:42:41 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257664#comment-13257664
] 

Per Steffensen edited comment on SOLR-3178 at 4/19/12 6:42 PM:
---------------------------------------------------------------

I dont care about credit :-)

It is just that I put myself on as Assigned to indicate that I was working on it. And I have
been - with a pause of a few weeks because my product owner prioritized me to look at something
different (money talkes - you know :-) ). I am soon ready to provide a full solution so why
not wait for that. My solution is a little bit different than yours (just had a very short
peek at your patch) but basically it is doing the same thing, but there is no need for both
your and my solution. I really hope you will wait until you see my contribution - it is much
more complete than what you have done until now, and we are already using it successfully
in a highly concurrent pilot-production environment.

My contribution solves at least a few problems that yours doesnt seem to solve (yet):
- It (also) works for multi-document update request, providing fully detailed feedback to
the client about which document-updates failed (and why) and which succeeded. I believe your
solution will just result in a 409 HTTP response if just one of many documents in the same
request failed, leaving no information to the user about which of the other documents in the
request where successfully handled (and stored), and if more of the documents (comming after
the on that did fail) would also have failed.
- It works (have been tested by heavily concurrent tests) in highly concurrent environments.
Looking shortly at you patch, I belive, that if two threads in the server JVM overlaps in
a way that is unfortunate enough, then it is possible that data will not be stored or will
be overwritten without an error being sent back to the client indicating that.
- Feedback to client is even possible if using ConcurrentSolrServer, because it has been changed
to provide a Future from "request" - a Future that will eventually provide the calling client
with the result from the request.

I understand your point about doing this incrementally, and I have done that internally. But
since I am not a Solr committer it is hard to push too many small partly-done features. If
I had been a committer you would have been able to follow the progress of my work more closely.
But fact is, that I have a full and well tested solution that I would like to provide as a
contribution very soon. I really hope you will wait for that, before committing a different
partly done solution to the same problems. Thanks a lot!

I would much rather want you to carefully read http://wiki.apache.org/solr/Per%20Steffensen/Update%20semantics
and provide feedback and comments. I will deliver everything described there (and more) very
soon. And then tell me how to ajust remaining time on this issue :-)

Regards, Per Steffensen 

                
      was (Author: steff1193):
    I dont care about credit :-)

It is just that I put myself on as Assigned to indicate that I was working on it. And I have
been - with a pause of a few weeks because my product owner prioritized me to look at something
different (money talkes - you know :-) ). I am soon ready to provide a full solution so why
not wait for that. My solution is a little bit different than yours (just had a very short
peek at you patch) but basically it is doing the same thing, but there is no need for both
you and my solution. I really hope you will wait until you see my contribution - it is much
more complete than what you have done until now, and we are already using it with success
in a highly concurrent pilot-production environment.

My contribution solves at least a few problems that yours doesnt seem to solve (yet):
- It works with multi-document update request, providing fully detailed feedback to the client
about which document-updates failed (and why) and which succeeded. I believe your solution
will just result in a 409 HTTP response if just one of many documents in the same request
failed, leaving no information to the user about which of the other documents in the request
where successfully handled, and if more of the documents (comming after the on that did fail)
would also have failed.
- It works (have been tested by heavily concurrent tests) in highly concurrent environments.
Looking shortly at you patch, I belive, that if two threads in the server JVM overlaps in
a way that is unfortunate enough, then it is possible that data will not be stored or will
be overwritten without an exception being thrown to indicate that to the client.

I understand your point about doing this incrementally, and I have done that internally. But
since I am not a Solr committer it is hard to push too many small partly-done features. If
I had been a committer you would have been able to follow the progress of my work more closely.
But fact is, that I have a full and well tested solution that I would like to provide as a
contribution very soon. I really hope you will wait for that, before committing different
a partly done solution to the same problems. Thanks a lot!

I would much rather want you to carefully read http://wiki.apache.org/solr/Per%20Steffensen/Update%20semantics
and provide feedback and comments. I will deliver everything described there (and more) very
soon.

Regards, Per Steffensen 

                  
> Versioning - optimistic locking
> -------------------------------
>
>                 Key: SOLR-3178
>                 URL: https://issues.apache.org/jira/browse/SOLR-3178
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>    Affects Versions: 3.5
>         Environment: All
>            Reporter: Per Steffensen
>            Assignee: Per Steffensen
>              Labels: RDBMS, insert, locking, nosql, optimistic, uniqueKey, update, versioning
>             Fix For: 4.0
>
>         Attachments: SOLR-3178.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> In order increase the ability of Solr to be used as a NoSql database (lots of concurrent
inserts, updates, deletes and queries in the entire lifetime of the index) instead of just
a search index (first: everything indexed (in one thread), after: only queries), I would like
Solr to support versioning to be used for optimistic locking.
> When my intent (see SOLR-3173) is to update an existing document, I will need to provide
a version-number equal to "the version number I got when I fetched the existing document for
update" plus one. If this provided version-number does not correspond to "the newest version-number
of that document at the time of update" plus one, I will get a VersionConflict error. If it
does correspond the document will be updated with the new one, so that "the newest version-number
of that document" is NOW one higher than before the update. Correct but efficient concurrency
handling.
> When my intent (see SOLR-3173) is to insert a new document, the version number provided
will not be used - instead a version-number 0 will be used. According to SOLR-3173 insert
will only succeed if a document with the same value on uniqueKey-field does not already exist.
> In general when talking about different versions of "the same document", of course we
need to be able to identify when a document "is the same" - that, per definition, is when
the values of the uniqueKey-fields are equal. 
> The functionality provided by this issue is only really meaningfull when you run with
"updateLog" activated.
> This issue might be solved more or less at the same time as SOLR-3173, and only one single
SVN patch might be given to cover both issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message