spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: query planner design doc?
Date Fri, 23 Jan 2015 20:25:36 GMT
No, are you looking for something in particular?

On Fri, Jan 23, 2015 at 9:44 AM, Nicholas Murphy <halcyonic@gmail.com>
wrote:

> Okay, thanks.  The design document mostly details the infrastructure for
> optimization strategies but doesn’t detail the strategies themselves.  I
> take it the set of strategies are basically embodied in
> SparkStrategies.scala...is there a design doc/roadmap/JIRA issue detailing
> what strategies exist and which are planned?
>
> Thanks,
> Nick
>
> On Jan 22, 2015, at 7:45 PM, Michael Armbrust <michael@databricks.com>
> wrote:
>
> Here is the initial design document for catalyst :
>
> https://docs.google.com/document/d/1Hc_Ehtr0G8SQUg69cmViZsMi55_Kf3tISD9GPGU5M1Y/edit
>
> Strategies (many of which are in SparkStragegies.scala) are the part that
> creates the physical operators from a catalyst logical plan.  These
> operators have execute() methods that actually call RDD operations.
>
> On Thu, Jan 22, 2015 at 3:19 PM, Nicholas Murphy <halcyonic@gmail.com>
> wrote:
>
>> Hi-
>>
>> Quick question: is there a design doc (or something more than “look at
>> the code”) for the query planner for Spark SQL (i.e., the component that
>> takes…Catalyst?…operator trees and translates them into SPARK operations)?
>>
>> Thanks,
>> Nick
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message