carbondata-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravipes...@apache.org
Subject carbondata git commit: [HOTFIX]Fixed Join Query Performance issue
Date Sun, 26 Aug 2018 17:39:29 GMT
Repository: carbondata
Updated Branches:
  refs/heads/master 17a4b485c -> f81543e95


[HOTFIX]Fixed Join Query Performance issue

Problem
Join Query is taking for time in case of carbon as for some of the Join Query it is not reusing
the exchange plan(Scanning+Snuffle) this is because in ReuseExchange it checking same result
of two plan is same or not, in case of carbon it always giving false as SparkCarbonTable equals
method is not overridden and comparison is failing.

Solution: Added equals method in SparkCarbonTableFormat

Tested TPCH query
Query:
select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice, sum(l_quantity) from customer,
orders, lineitem where o_orderkey in ( select l_orderkey from lineitem group by l_orderkey
having sum(l_quantity) > 300 ) and c_custkey = o_custkey and o_orderkey = l_orderkey group
by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice order by o_totalprice desc, o_orderdate;

This closes #2650


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/f81543e9
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/f81543e9
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/f81543e9

Branch: refs/heads/master
Commit: f81543e958f89a79d20de9d02cc98be42a8a06b5
Parents: 17a4b48
Author: kumarvishal09 <kumarvishal1802@gmail.com>
Authored: Thu Aug 23 14:00:55 2018 +0530
Committer: ravipesala <ravi.pesala@gmail.com>
Committed: Sun Aug 26 23:09:19 2018 +0530

----------------------------------------------------------------------
 .../spark/sql/execution/datasources/SparkCarbonTableFormat.scala    | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/carbondata/blob/f81543e9/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala
----------------------------------------------------------------------
diff --git a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala
b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala
index 0d62438..6716707 100644
--- a/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala
+++ b/integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala
@@ -221,6 +221,7 @@ with Serializable {
 
     }
   }
+  override def equals(other: Any): Boolean = other.isInstanceOf[SparkCarbonTableFormat]
 }
 
 case class CarbonSQLHadoopMapReduceCommitProtocol(jobId: String, path: String, isAppend:
Boolean)


Mime
View raw message