spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng, Hao" <hao.ch...@intel.com>
Subject RE: Join Order Optimization
Date Mon, 12 Oct 2015 02:27:46 GMT
Yes, I think the SPARK-2211 should be the right place to follow the CBO stuff, but probably
that will not happen right away.

The jira issue introduce the statistic info can be found at:
https://issues.apache.org/jira/browse/SPARK-2393

Hao

From: Raajay [mailto:raajay.v@gmail.com]
Sent: Monday, October 12, 2015 10:17 AM
To: Cheng, Hao
Cc: user@spark.apache.org
Subject: Re: Join Order Optimization

Hi Cheng,
Could you point me to the JIRA that introduced this change ?

Also, is this SPARK-2211 the right issue to follow for cost-based optimization?
Thanks
Raajay


On Sun, Oct 11, 2015 at 7:57 PM, Cheng, Hao <hao.cheng@intel.com<mailto:hao.cheng@intel.com>>
wrote:
Spark SQL supports very basic join reordering optimization, based on the raw table data size,
this was added couple major releases back.

And the “EXPLAIN EXTENDED query” command is a very informative tool to verify whether
the optimization taking effect.

From: Raajay [mailto:raajay.v@gmail.com<mailto:raajay.v@gmail.com>]
Sent: Sunday, October 11, 2015 9:22 AM
To: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Join Order Optimization

Hello,
Does Spark-SQL support join order optimization as of the 1.5.1 release ? From the release
notes, I did not see support for this feature, but figured will ask the users-list to be sure.
Thanks
Raajay

Mime
View raw message