olingo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ralf Handl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OLINGO-1087) Precision in EdmDecimal not calculated correctly
Date Thu, 02 Mar 2017 09:35:46 GMT

    [ https://issues.apache.org/jira/browse/OLINGO-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891918#comment-15891918
] 

Ralf Handl commented on OLINGO-1087:
------------------------------------

OData V2's data type Edm.Decimal follows the rules for the SQL type DECIMAL(p,s) or NUMBER(p,s),
see e.g. https://msdn.microsoft.com/en-us/library/ms190476.aspx. It is a decimally stored
fixed-point number, see https://en.wikipedia.org/wiki/Fixed-point_arithmetic.

Precision in this context is the length of the decimal mantissa, and Scale is the negative
exponent of the base-10 scaling factor. This means that you have to take both Precision and
Scale into account in this discussion. 

For the rather lengthy number above you'd need Precision=35 and Scale=35 to store it losslessly
in a DECIMAL column of a SQL database, limiting you to storing numbers that are greater than
-1 and less than 1 with 35 decimal digits to the right of the decimal point. 

Example 17 in http://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/csprd01/odata-csdl-xml-v4.01-csprd01.html#_Toc472333020
explains this for the more readable case of Precision=2 and Scale=2.

> Precision in EdmDecimal not calculated correctly
> ------------------------------------------------
>
>                 Key: OLINGO-1087
>                 URL: https://issues.apache.org/jira/browse/OLINGO-1087
>             Project: Olingo
>          Issue Type: Bug
>          Components: odata2-core
>    Affects Versions: V2 2.0.8
>            Reporter: Svante von Erichsen
>         Attachments: 0001-OLINGO-1087-Fix-calculation-of-precision.patch
>
>
> The "precision" found in a read number is currently at least the number of decimals (omitting
trailing zeroes).  That seems to be wrong, or at least not consistent with at least some server
side view of the matter, for numbers smaller than 0.1 (the number shown has 35 decimals, but
precision 34):
> org.apache.olingo.odata2.api.edm.EdmSimpleTypeException: The metadata constraints '[Precision=34]'
do not match the literal '0.08102359900000000170194880411145277'.
>                 at org.apache.olingo.odata2.core.edm.EdmDecimal.internalValueOfString(EdmDecimal.java:107)
>                 at org.apache.olingo.odata2.core.edm.AbstractSimpleType.valueOfString(AbstractSimpleType.java:91)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readSimpleProperty(JsonPropertyConsumer.java:236)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readPropertyValue(JsonPropertyConsumer.java:169)
>                 ... 69 common frames omitted
> Wrapped by: org.apache.olingo.odata2.api.ep.EntityProviderException: An exception of
type 'EdmSimpleTypeException' occurred.
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readPropertyValue(JsonPropertyConsumer.java:171)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntryConsumer.handleName(JsonEntryConsumer.java:172)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntryConsumer.readEntryContent(JsonEntryConsumer.java:130)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntryConsumer.readFeedEntry(JsonEntryConsumer.java:117)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readArrayContent(JsonFeedConsumer.java:153)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.handleName(JsonFeedConsumer.java:122)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readFeedContent(JsonFeedConsumer.java:111)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readFeed(JsonFeedConsumer.java:96)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readFeedStandalone(JsonFeedConsumer.java:63)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntityConsumer.readDeltaFeed(JsonEntityConsumer.java:95)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntityConsumer.readFeed(JsonEntityConsumer.java:81)
>                 at org.apache.olingo.odata2.core.ep.JsonEntityProvider.readFeed(JsonEntityProvider.java:309)
>                 at org.apache.olingo.odata2.core.ep.ProviderFacadeImpl.readFeed(ProviderFacadeImpl.java:165)
>                 at org.apache.olingo.odata2.api.ep.EntityProvider.readFeed(EntityProvider.java:676)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message