spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yifei Wu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-24468) DecimalType `adjustPrecisionScale` might fail when scale is negative
Date Tue, 05 Jun 2018 09:23:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-24468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yifei Wu updated SPARK-24468:
-----------------------------
    Summary: DecimalType `adjustPrecisionScale` might fail when scale is negative  (was: DecimalType
`adjustPrecisionScale` might fail when assert(scale>=0))

> DecimalType `adjustPrecisionScale` might fail when scale is negative
> --------------------------------------------------------------------
>
>                 Key: SPARK-24468
>                 URL: https://issues.apache.org/jira/browse/SPARK-24468
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.3.0
>            Reporter: Yifei Wu
>            Priority: Minor
>
> Hi, I am using MySQL JDBC Driver along with Spark to do some sql queries.
> When multiplying a LongType with a decimal in scientific notation, say
> {code}
> spark.sql("select some_int * 2.34E10 from t").show{code}
> , decimal 2.34E10 will be treated as decimal(3,-8), and some_int will be casted as decimal(20,0).
> So according to the rules in comments:
> {code:java}
> /*
>  *   Operation    Result Precision                        Result Scale
>  *   ------------------------------------------------------------------------
>  *   e1 + e2      max(s1, s2) + max(p1-s1, p2-s2) + 1     max(s1, s2)
>  *   e1 - e2      max(s1, s2) + max(p1-s1, p2-s2) + 1     max(s1, s2)
>  *   e1 * e2      p1 + p2 + 1                             s1 + s2
>  *   e1 / e2      p1 - s1 + s2 + max(6, s1 + p2 + 1)      max(6, s1 + p2 + 1)
>  *   e1 % e2      min(p1-s1, p2-s2) + max(s1, s2)         max(s1, s2)
>  *   e1 union e2  max(s1, s2) + max(p1-s1, p2-s2)         max(s1, s2)
> */
> {code}
> their multiplication will be decimal(3+20+1,-8+0) and thus fails the assert assumption
(scale>=0) on DecimalType.scala:166.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message