Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Thu, 3 May 2012 14:32:51 +0000 (UTC)
From: "Per Steffensen (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <373748711.21965.1336055571068.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <425759066.2569.1330517877492.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Issue Comment Edited] (SOLR-3178) Versioning - optimistic
 locking
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/SOLR-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267463#comment-13267463 ] 

Per Steffensen edited comment on SOLR-3178 at 5/3/12 2:32 PM:
--------------------------------------------------------------

Find attached SOLR-3173_3178_3382_3428_plus.patch

The patch fits on top of trunk revision 1332666 and is ready for commit.

Since last patch (SOLR_3173_3178_3382_plus.patch) I have made the following:
* Implemented test ClassicConsistencyHybridUpdateSemanticsSolrCloudTest verifying that partial errors are propagated correctly in a cloud (ZK) setup, when an update document is not sent directly to the Solr instance running the leader of the slice where the document is to be stored, and the Solr instance therefore has to forward the document to the leader, and when the leader forwards to replica. This made me discover the problems discribed in SOLR-3428, which I had to fix in order to make the test green.
* Implemented tests (in JsonLoaderTest and CSVRequestHandlerTest) verifying that you can send part references as claimed on http://wiki.apache.org/solr/Per%20Steffensen/Update%20semantics when you are sending JSON or CSV requests
* Documents for which a partial error occur on leader will not be forwarded to replica.
* Partial errors are now catched and collected for the entire request in DistributedUpdateProcessor, but the actual version-check etc is still performed in DirectUpdateHandler2 (because that is an per-document-level task)
* Added Apache License 2.0 text to all new files
* Added class-level-JavaDoc to all new classes 
* Removed a few unimportant changes

I will be leaving for an extended weekend now, and will not be available again before wedensday next week. I will probably read and answer comments from time to time until saturday night, though.

I really really really hope to return to the great news about the patch having been committed. I believe it is now really ready - well worked through features, well covered by tests and all existing plus new tests are green. Great progress, maybe not perfection, but we can shape the last edges later if we find any. Hope you will take the opportunity to commit before too long, now that you have a patch based on a fairly new revision, to avoid having to many conflicts to solve.

Just to avoid any mistakes - the patch covers SOLR-3173, SOLR-3178, SOLR-3382 and SOLR-3428.

Regards, Per Steffensen
                
      was (Author: steff1193):
    Find attached SOLR-3173_3178_3382_3428_plus.patch

The patch fits on top of trunk revision 1332666 and is ready for commit.

Since last patch (SOLR_3173_3178_3382_plus.patch) I have made the following:
* Implemented test ClassicConsistencyHybridUpdateSemanticsSolrCloudTest verifying that partial errors are propagated correctly in a cloud (ZK) setup, when an update document is not sent directly to the Solr instance running the leader of the slice where the document is to be stored, and the Solr instance therefore has to forward the document to the leader, and when the leader forwards to replica. This made me discover the problems discribed in SOLR-3428, which I had to fix in order to make the test green.
* Implemented tests (in JsonLoaderTest and CSVRequestHandlerTest) verifying that you can send part references as claimed on http://wiki.apache.org/solr/Per%20Steffensen/Update%20semantics when you are sending JSON or CSV requests
* Documents for which a partial error occur on leader will not be forwarded to replica.
* Partial errors are now catched and collected for the entire request in DistributedUpdateProcessor, but the actual version-check etc is still performed in DirectUpdateHandler2 (because that is an per-document-level task)
* Added Apache License 2.0 text to all new files
* Added class-level-JavaDoc to all new classes 
* Removed a few unimportant changes

I will be leaving for an extended weekend now, and will not be available again before wedensday next week. I will probably read and answer comments from time to time until saturday night, though.

I really really really hope to return to the great news about the patch having been committed. I believe it is now really ready - well worked through features, well covered by tests and all existing plus new tests are green. Great progress, maybe not perfection, but we can shape the last edges later if we find any. Hope you will take the opportunity to commit before too long, now that you have a patch based on a fairly new revision, to avoid having to many conflicts to solve.

Just to avoid any mistakes - the patch covers SORL-3173 , SORL-3178 , SOLR-3382 and SOLR-3428.

Regards, Per Steffensen
                  
> Versioning - optimistic locking
> -------------------------------
>
>                 Key: SOLR-3178
>                 URL: https://issues.apache.org/jira/browse/SOLR-3178
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>    Affects Versions: 3.5
>         Environment: All
>            Reporter: Per Steffensen
>            Assignee: Per Steffensen
>              Labels: RDBMS, insert, locking, nosql, optimistic, uniqueKey, update, versioning
>             Fix For: 4.0
>
>         Attachments: SOLR-3173_3178_3382_3428_plus.patch, SOLR-3178.patch, SOLR_3173_3178_3382_plus.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> In order increase the ability of Solr to be used as a NoSql database (lots of concurrent inserts, updates, deletes and queries in the entire lifetime of the index) instead of just a search index (first: everything indexed (in one thread), after: only queries), I would like Solr to support versioning to be used for optimistic locking.
> When my intent (see SOLR-3173) is to update an existing document, I will need to provide a version-number equal to "the version number I got when I fetched the existing document for update" plus one. If this provided version-number does not correspond to "the newest version-number of that document at the time of update" plus one, I will get a VersionConflict error. If it does correspond the document will be updated with the new one, so that "the newest version-number of that document" is NOW one higher than before the update. Correct but efficient concurrency handling.
> When my intent (see SOLR-3173) is to insert a new document, the version number provided will not be used - instead a version-number 0 will be used. According to SOLR-3173 insert will only succeed if a document with the same value on uniqueKey-field does not already exist.
> In general when talking about different versions of "the same document", of course we need to be able to identify when a document "is the same" - that, per definition, is when the values of the uniqueKey-fields are equal. 
> The functionality provided by this issue is only really meaningfull when you run with "updateLog" activated.
> This issue might be solved more or less at the same time as SOLR-3173, and only one single SVN patch might be given to cover both issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org