lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marc Portier (JIRA)" <>
Subject [jira] [Created] (SOLR-6364) _version_ value too big for javascript clients causing reported _version_ never matching internal _version_ ==> suggested resolution: json should communicate _version_ as string!
Date Mon, 11 Aug 2014 20:44:13 GMT
Marc Portier created SOLR-6364:

             Summary: _version_ value too big for javascript clients causing reported _version_
never matching internal _version_ ==> suggested resolution: json should communicate _version_
as string! 
                 Key: SOLR-6364
             Project: Solr
          Issue Type: Bug
          Components: update
    Affects Versions: 4.9
         Environment: ubuntu 14.04 desktop 32bit

Oracle Corporation Java HotSpot(TM) Server VM (1.7.0_45 24.45-b08)

lucene-spec 4.9.0
lucene-impl 4.9.0 1604085 - rmuir - 2014-06-20 06:22:23

solr-spec  4.9.0
solr-impl  4.9.0 1604085 - rmuir - 2014-06-20 06:34:03

            Reporter: Marc Portier

There seems to be a 100 based rounding active in the output/rendition/return of the _version_
field of added documents.  Internally on the solr side however the real number (non-rounded)
is effective and introduces conflicts with the optimistic concurrency logic.

Apparently this is to be expected in all Javascript clients, since the _version_ numbers used
are too big to fit into Javascript Number variables without loss of precision.

Here is what one can do to see this in action - all steps below with 
1/ using the solr4 admin UI on 
2/ the request-handler box set to 
3/ by adding the following into the "documents"  section on the page:

[1] for create 

{ "id": "tst-abcd, "version": 1, "type": "test", "title": ["title"], "_version_": -1 }


{  "responseHeader": {    "status": 0,    "QTime": 1882  },
  "adds": [    "tst-abcd",    1476172747866374100  ]

>> see the returned _version_ is a multiple of 100, always!

[2] update

{ "id": "tst-abcd", "version": 2, "type": "test", "title": ["title update"], "_version_":
1476172747866374100 }

Response Error:
{  "responseHeader": {    "status": 409,    "QTime": 51  },
  "error": {    "msg": "version conflict for tst-abcd expected=1476172747866374100 actual=1476172747866374144",
    "code": 409  }}

>> notice how the error-message-string correctly mentions the real actual _version_
that is effective (not rounded to 100)

[3] corrected update, using that effective number

{ "id": "tst-abcd", "version": 2, "type": "test", "title": ["title update"], "_version_":
1476172747866374144 }


{  "responseHeader": {    "status": 0,    "QTime": 597  },
  "adds": [    "tst-abcd",    1476173026894545000  ] }

Odd at first this behaviour is not shown with curl on the command line...

[1] create

$ curl "$solrbase/update?commit=true&versions=true" -H 'Content-type:application/json'
-d '[{ "id": "tst-1234", "version": 1, "type": "test", "title": ["title"], "_version_": -1



>> number is not rounded, looks good!

[2] update 

$ curl "$solrbase/update?commit=true&versions=true" -H 'Content-type:application/json'
-d '[{ "id": "tst-1234", "version": 2, "type": "test", "title": ["title updated"], "_version_":
1476163269470191616 }]'



All this was pretty much a mistery to me untill I came across this:

This looks like passing down the too big numbers in the _version_ as strings should avoid
the issue. Or use numbers that aren't that big, since apparently: "The largest number JavaScript
can handle without loss of precision is 9007199254740992"  -- quoted from that stackoverflow

There are more references (below) talking about this being a Javascript limitation rather
then a pure json-spec issue, nevertheless... it might be easier to adapt solr to deal with
this know Javascript limitation and thus helping out the Javascript clients out there?


In terms of backwards compatibility I don't see an easy way out for the moment.  
- clients that expect _version_ to be numeric might not handle the string
- in existing deployments it might be hard to reduce all the already existing _version_ to
stay undere the limit...

I still have to investigate into receiving and parsing XML replies from SOLR instead - making
sure I keep the returned _version_ info in a Javascript string.  Hoping that might work as
a timely (but not as elegant) workaround.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message