spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-25415) Make plan change log in RuleExecutor configurable by SQLConf
Date Wed, 12 Sep 2018 22:07:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-25415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612804#comment-16612804
] 

Apache Spark commented on SPARK-25415:
--------------------------------------

User 'dilipbiswal' has created a pull request for this issue:
https://github.com/apache/spark/pull/22407

> Make plan change log in RuleExecutor configurable by SQLConf
> ------------------------------------------------------------
>
>                 Key: SPARK-25415
>                 URL: https://issues.apache.org/jira/browse/SPARK-25415
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.3.1
>            Reporter: Maryann Xue
>            Priority: Minor
>
> In RuleExecutor, after applying a rule, if the plan has changed, the before and after
plan will be logged using level "trace". At times, however, such information can be very helpful
for debugging, so making the log level configurable in SQLConf would allow users to turn on
the plan change log independently and save the trouble of tweaking log4j settings.
> Meanwhile, filtering plan change log for specific rules can also be very useful.
> So I propose adding two confs:
> 1. spark.sql.optimizer.planChangeLog.level - set a specific log level for logging plan
changes after a rule is applied.
> 2. spark.sql.optimizer.planChangeLog.rules - enable plan change logging only for a set
of specified rules, separated by commas.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message