spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Youssef Hatem (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-6320) Adding new query plan strategy to SQLContext
Date Tue, 17 Mar 2015 15:10:38 GMT

    [ https://issues.apache.org/jira/browse/SPARK-6320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365274#comment-14365274
] 

Youssef Hatem commented on SPARK-6320:
--------------------------------------

Thank you Michael for your response,

Actually I am trying to extend SQL syntax with custom constructs which would require altering
the workflow of {{SparkSQL}} all the way starting from the lexer and ending with the optimization
and physical planning.

I thought this should be possible without having to extend many classes. However not being
able to use {{planLater}} forces me to do seemingly unnecessary extension of {{SparkPlanner}}
specially that there seem to be some logic to handle these scenarios (i.e. the {{extraStrategies}}
sequence).

> Adding new query plan strategy to SQLContext
> --------------------------------------------
>
>                 Key: SPARK-6320
>                 URL: https://issues.apache.org/jira/browse/SPARK-6320
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.3.0
>            Reporter: Youssef Hatem
>            Priority: Minor
>
> Hi,
> I would like to add a new strategy to {{SQLContext}}. To do this I created a new class
which extends {{Strategy}}. In my new class I need to call {{planLater}} function. However
this method is defined in {{SparkPlanner}} (which itself inherits the method from {{QueryPlanner}}).
> To my knowledge the only way to make {{planLater}} function visible to my new strategy
is to define my strategy inside another class that extends {{SparkPlanner}} and inherits {{planLater}}
as a result, by doing so I will have to extend the {{SQLContext}} such that I can override
the {{planner}} field with the new {{Planner}} class I created.
> It seems that this is a design problem because adding a new strategy seems to require
extending {{SQLContext}} (unless I am doing it wrong and there is a better way to do it).
> Thanks a lot,
> Youssef



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message