spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yh...@apache.org
Subject spark git commit: [SPARK-17102][SQL] bypass UserDefinedGenerator for json format check
Date Wed, 17 Aug 2016 16:31:45 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-2.0 22c7660a8 -> 394d59866


[SPARK-17102][SQL] bypass UserDefinedGenerator for json format check

## What changes were proposed in this pull request?

We use reflection to convert `TreeNode` to json string, and currently don't support arbitrary
object. `UserDefinedGenerator` takes a function object, so we should skip json format test
for it, or the tests can be flacky, e.g. `DataFrameSuite.simple explode`, this test always
fail with scala 2.10(branch 1.6 builds with scala 2.10 by default), but pass with scala 2.11(master
branch builds with scala 2.11 by default).

## How was this patch tested?

N/A

Author: Wenchen Fan <wenchen@databricks.com>

Closes #14679 from cloud-fan/json.

(cherry picked from commit 928ca1c6d12b23d84f9b6205e22d2e756311f072)
Signed-off-by: Yin Huai <yhuai@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/394d5986
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/394d5986
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/394d5986

Branch: refs/heads/branch-2.0
Commit: 394d5986617e65852422afeb8d755e38795bbe25
Parents: 22c7660
Author: Wenchen Fan <wenchen@databricks.com>
Authored: Wed Aug 17 09:31:22 2016 -0700
Committer: Yin Huai <yhuai@databricks.com>
Committed: Wed Aug 17 09:31:41 2016 -0700

----------------------------------------------------------------------
 sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/394d5986/sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala b/sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
index e8480a7..b2c051d 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
@@ -242,9 +242,10 @@ abstract class QueryTest extends PlanTest {
       case p if p.getClass.getSimpleName == "MetastoreRelation" => return
       case _: MemoryPlan => return
     }.transformAllExpressions {
-      case a: ImperativeAggregate => return
+      case _: ImperativeAggregate => return
       case _: TypedAggregateExpression => return
       case Literal(_, _: ObjectType) => return
+      case _: UserDefinedGenerator => return
     }
 
     // bypass hive tests before we fix all corner cases in hive module.


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message