# olingo-dev mailing list archives

##### Site index · List index
Message view
Top
From "Ralf Handl (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (OLINGO-1087) Precision in EdmDecimal not calculated correctly
Date Thu, 02 Mar 2017 10:43:45 GMT
```
[ https://issues.apache.org/jira/browse/OLINGO-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892031#comment-15892031
]

Ralf Handl edited comment on OLINGO-1087 at 3/2/17 10:43 AM:
-------------------------------------------------------------

The default of 0 was specified in OData V2:

{quote}
2.2.1.7 Decimal
The Decimal type represents numeric values with fixed precision and scale. The required precision
and scale can be specified using its optional Precision and Scale facets. This type can describe
a numeric value ranging from negative 10^255 + 1 to positive 10^255 -1.

2.2.1.7.1 Facets

2.2.1.7.1.1 Precision
This is a positive integer that specifies the maximum number of decimal digits that an instance
of the decimal type can have, both to the left and to the right of the decimal point. Possible
values for Precision are 1, 2, or 3.

2.2.1.7.1.2 Scale
This is a positive integer that specifies the maximum number of decimal digits to the right
of the decimal point that an instance of this type can have. The Scale value can range from
0 through the specified Precision value. The default Scale is 0.
{quote}

was (Author: ralfhandl):
The default of 0 was specified in OData V2:

2.2.1.7 Decimal
The Decimal type represents numeric values with fixed precision and scale. The required precision
and scale can be specified using its optional Precision and Scale facets. This type can describe
a numeric value ranging from negative 10^255 + 1 to positive 10^255 -1.

2.2.1.7.1 Facets

2.2.1.7.1.1 Precision
This is a positive integer that specifies the maximum number of decimal digits that an instance
of the decimal type can have, both to the left and to the right of the decimal point. Possible
values for Precision are 1, 2, or 3.

2.2.1.7.1.2 Scale
This is a positive integer that specifies the maximum number of decimal digits to the right
of the decimal point that an instance of this type can have. The Scale value can range from
0 through the specified Precision value. The default Scale is 0.

> Precision in EdmDecimal not calculated correctly
> ------------------------------------------------
>
>                 Key: OLINGO-1087
>                 URL: https://issues.apache.org/jira/browse/OLINGO-1087
>             Project: Olingo
>          Issue Type: Bug
>          Components: odata2-core
>    Affects Versions: V2 2.0.8
>            Reporter: Svante von Erichsen
>         Attachments: 0001-OLINGO-1087-Fix-calculation-of-precision.patch
>
>
> The "precision" found in a read number is currently at least the number of decimals (omitting
trailing zeroes).  That seems to be wrong, or at least not consistent with at least some server
side view of the matter, for numbers smaller than 0.1 (the number shown has 35 decimals, but
precision 34):
> org.apache.olingo.odata2.api.edm.EdmSimpleTypeException: The metadata constraints '[Precision=34]'
do not match the literal '0.08102359900000000170194880411145277'.
>                 at org.apache.olingo.odata2.core.edm.EdmDecimal.internalValueOfString(EdmDecimal.java:107)
>                 at org.apache.olingo.odata2.core.edm.AbstractSimpleType.valueOfString(AbstractSimpleType.java:91)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readSimpleProperty(JsonPropertyConsumer.java:236)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readPropertyValue(JsonPropertyConsumer.java:169)
>                 ... 69 common frames omitted
> Wrapped by: org.apache.olingo.odata2.api.ep.EntityProviderException: An exception of
type 'EdmSimpleTypeException' occurred.
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonPropertyConsumer.readPropertyValue(JsonPropertyConsumer.java:171)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntryConsumer.handleName(JsonEntryConsumer.java:172)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntryConsumer.readEntryContent(JsonEntryConsumer.java:130)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntryConsumer.readFeedEntry(JsonEntryConsumer.java:117)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readArrayContent(JsonFeedConsumer.java:153)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.handleName(JsonFeedConsumer.java:122)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readFeedContent(JsonFeedConsumer.java:111)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readFeed(JsonFeedConsumer.java:96)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonFeedConsumer.readFeedStandalone(JsonFeedConsumer.java:63)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntityConsumer.readDeltaFeed(JsonEntityConsumer.java:95)
>                 at org.apache.olingo.odata2.core.ep.consumer.JsonEntityConsumer.readFeed(JsonEntityConsumer.java:81)
>                 at org.apache.olingo.odata2.core.ep.JsonEntityProvider.readFeed(JsonEntityProvider.java:309)
>                 at org.apache.olingo.odata2.core.ep.ProviderFacadeImpl.readFeed(ProviderFacadeImpl.java:165)
>                 at org.apache.olingo.odata2.api.ep.EntityProvider.readFeed(EntityProvider.java:676)

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

```
Mime
View raw message