lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marc Portier (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SOLR-6364) _version_ value too big for javascript clients causing reported _version_ never matching internal _version_ ==> suggested resolution: json should communicate _version_ as string!
Date Mon, 11 Aug 2014 20:44:13 GMT
Marc Portier created SOLR-6364:
----------------------------------

             Summary: _version_ value too big for javascript clients causing reported _version_
never matching internal _version_ ==> suggested resolution: json should communicate _version_
as string! 
                 Key: SOLR-6364
                 URL: https://issues.apache.org/jira/browse/SOLR-6364
             Project: Solr
          Issue Type: Bug
          Components: update
    Affects Versions: 4.9
         Environment: ubuntu 14.04 desktop 32bit

Oracle Corporation Java HotSpot(TM) Server VM (1.7.0_45 24.45-b08)

lucene-spec 4.9.0
lucene-impl 4.9.0 1604085 - rmuir - 2014-06-20 06:22:23

solr-spec  4.9.0
solr-impl  4.9.0 1604085 - rmuir - 2014-06-20 06:34:03

            Reporter: Marc Portier


There seems to be a 100 based rounding active in the output/rendition/return of the _version_
field of added documents.  Internally on the solr side however the real number (non-rounded)
is effective and introduces conflicts with the optimistic concurrency logic.

Apparently this is to be expected in all Javascript clients, since the _version_ numbers used
are too big to fit into Javascript Number variables without loss of precision.

Here is what one can do to see this in action - all steps below with 
1/ using the solr4 admin UI on 
    http://localhost:8983/solr/#/mycore/documents
2/ the request-handler box set to 
    /update?commit=true&versions=true
3/ by adding the following into the "documents"  section on the page:

[1] for create 
Using:

{ "id": "tst-abcd, "version": 1, "type": "test", "title": ["title"], "_version_": -1 }

Response:

{  "responseHeader": {    "status": 0,    "QTime": 1882  },
  "adds": [    "tst-abcd",    1476172747866374100  ]
}

>> see the returned _version_ is a multiple of 100, always!


[2] update
Using:

{ "id": "tst-abcd", "version": 2, "type": "test", "title": ["title update"], "_version_":
1476172747866374100 }

Response Error:
{  "responseHeader": {    "status": 409,    "QTime": 51  },
  "error": {    "msg": "version conflict for tst-abcd expected=1476172747866374100 actual=1476172747866374144",
    "code": 409  }}

>> notice how the error-message-string correctly mentions the real actual _version_
that is effective (not rounded to 100)


[3] corrected update, using that effective number

{ "id": "tst-abcd", "version": 2, "type": "test", "title": ["title update"], "_version_":
1476172747866374144 }

Response:

{  "responseHeader": {    "status": 0,    "QTime": 597  },
  "adds": [    "tst-abcd",    1476173026894545000  ] }



Odd at first this behaviour is not shown with curl on the command line...

[1] create

$ curl "$solrbase/update?commit=true&versions=true" -H 'Content-type:application/json'
-d '[{ "id": "tst-1234", "version": 1, "type": "test", "title": ["title"], "_version_": -1
}]'


response: 

{"responseHeader":{"status":0,"QTime":587},"adds":["tst-1234",1476163269470191616]}

>> number is not rounded, looks good!

[2] update 

$ curl "$solrbase/update?commit=true&versions=true" -H 'Content-type:application/json'
-d '[{ "id": "tst-1234", "version": 2, "type": "test", "title": ["title updated"], "_version_":
1476163269470191616 }]'


response: 

{"responseHeader":{"status":0,"QTime":512},"adds":["tst-1234",1476163320472928256]}



All this was pretty much a mistery to me untill I came across this:

http://stackoverflow.com/questions/15689790/parse-json-in-javascript-long-numbers-get-rounded


This looks like passing down the too big numbers in the _version_ as strings should avoid
the issue. Or use numbers that aren't that big, since apparently: "The largest number JavaScript
can handle without loss of precision is 9007199254740992"  -- quoted from that stackoverflow
page.

There are more references (below) talking about this being a Javascript limitation rather
then a pure json-spec issue, nevertheless... it might be easier to adapt solr to deal with
this know Javascript limitation and thus helping out the Javascript clients out there?

- http://stackoverflow.com/questions/307179/what-is-javascripts-max-int-whats-the-highest-integer-value-a-number-can-go-t
- http://stackoverflow.com/questions/13502398/json-integers-limit-on-size

In terms of backwards compatibility I don't see an easy way out for the moment.  
- clients that expect _version_ to be numeric might not handle the string
- in existing deployments it might be hard to reduce all the already existing _version_ to
stay undere the limit...

I still have to investigate into receiving and parsing XML replies from SOLR instead - making
sure I keep the returned _version_ info in a Javascript string.  Hoping that might work as
a timely (but not as elegant) workaround.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message