spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kazuaki Ishizaki (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt
Date Thu, 09 Aug 2018 17:06:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575137#comment-16575137
] 

Kazuaki Ishizaki edited comment on SPARK-25036 at 8/9/18 5:05 PM:
------------------------------------------------------------------

Another type of compilation error is found. Added the log to the description


was (Author: kiszk):
Another type of compilation error is found

> Scala 2.12 issues: Compilation error with sbt
> ---------------------------------------------
>
>                 Key: SPARK-25036
>                 URL: https://issues.apache.org/jira/browse/SPARK-25036
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.3.0, 2.4.0
>            Reporter: Kazuaki Ishizaki
>            Assignee: Kazuaki Ishizaki
>            Priority: Major
>             Fix For: 2.4.0
>
>
> When compiling with sbt, the following errors occur:
> There are two types:
> 1. {{ExprValue.isNull}} is compared with unexpected type.
> 1. {{match may not be exhaustive}} is detected at {{match}}
> The first one is more serious since it may also generate incorrect code in Spark 2.3.
> {code}
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/ValueInterval.scala:63:
match may not be exhaustive.
> [error] It would fail on the following inputs: (NumericValueInterval(_, _), _), (_, NumericValueInterval(_,
_)), (_, _)
> [error] [warn]   def isIntersected(r1: ValueInterval, r2: ValueInterval): Boolean = (r1,
r2) match {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/ValueInterval.scala:79:
match may not be exhaustive.
> [error] It would fail on the following inputs: (NumericValueInterval(_, _), _), (_, NumericValueInterval(_,
_)), (_, _)
> [error] [warn]     (r1, r2) match {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproxCountDistinctForIntervals.scala:67:
match may not be exhaustive.
> [error] It would fail on the following inputs: (ArrayType(_, _), _), (_, ArrayData()),
(_, _)
> [error] [warn]     (endpointsExpression.dataType, endpointsExpression.eval()) match {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:470:
match may not be exhaustive.
> [error] It would fail on the following inputs: NewFunctionSpec(_, None, Some(_)), NewFunctionSpec(_,
Some(_), None)
> [error] [warn]     newFunction match {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala:94:
org.apache.spark.sql.catalyst.expressions.codegen.ExprValue and String are unrelated: they
will most likely always compare unequal
> [error] [warn]         if (eval.isNull != "true") {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala:126:
org.apache.spark.sql.catalyst.expressions.codegen.ExprValue and String are unrelated: they
will most likely never compare equal
> [error] [warn]              if (eval.isNull == "true") {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala:133:
org.apache.spark.sql.catalyst.expressions.codegen.ExprValue and String are unrelated: they
will most likely never compare equal
> [error] [warn]             if (eval.isNull == "true") {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala:709:
match may not be exhaustive.
> [error] It would fail on the following input: Schema((x: org.apache.spark.sql.types.DataType
forSome x not in org.apache.spark.sql.types.StructType), _)
> [error] [warn]   def attributesFor[T: TypeTag]: Seq[Attribute] = schemaFor[T] match {
> [error] [warn] 
> [error] [warn] /home/ishizaki/Spark/PR/scala212/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala:90:
org.apache.spark.sql.catalyst.expressions.codegen.ExprValue and String are unrelated: they
will most likely never compare equal
> [error] [warn]       if (inputs.map(_.isNull).forall(_ == "false")) {
> [error] [warn] 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message