spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-22479) SaveIntoDataSourceCommand logs jdbc credentials
Date Fri, 02 Mar 2018 15:57:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383733#comment-16383733
] 

Thomas Graves commented on SPARK-22479:
---------------------------------------

Also the example above shows the password, but the password should have been already redacted,
this pr excluded url, user, and username. Was the password not being redacted for some reason?

> SaveIntoDataSourceCommand logs jdbc credentials
> -----------------------------------------------
>
>                 Key: SPARK-22479
>                 URL: https://issues.apache.org/jira/browse/SPARK-22479
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.2.0
>            Reporter: Onur Satici
>            Assignee: Onur Satici
>            Priority: Major
>             Fix For: 2.2.1, 2.3.0
>
>
> JDBC credentials are not redacted in plans including a 'SaveIntoDataSourceCommand'.
> Steps to reproduce:
> {code}
> spark-shell --packages org.postgresql:postgresql:42.1.1
> {code}
> {code}
> import org.apache.spark.sql.execution.QueryExecution
> import org.apache.spark.sql.util.QueryExecutionListener
> val listener = new QueryExecutionListener {
>   override def onFailure(funcName: String, qe: QueryExecution, exception: Exception):
Unit = {}
>   override def onSuccess(funcName: String, qe: QueryExecution, duration: Long): Unit
= {
>     System.out.println(qe.toString())
>   }
> }
> spark.listenerManager.register(listener)
> spark.range(100).write.format("jdbc").option("url", "jdbc:postgresql:sparkdb").option("password",
"pass").option("driver", "org.postgresql.Driver").option("dbtable", "test").save()
> {code}
> The above will yield the following plan:
> {code}
> == Parsed Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver,
url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
>    +- Range (0, 100, step=1, splits=Some(8))
> == Analyzed Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver,
url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
>    +- Range (0, 100, step=1, splits=Some(8))
> == Optimized Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver,
url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
>    +- Range (0, 100, step=1, splits=Some(8))
> == Physical Plan ==
> ExecutedCommand
>    +- SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver,
url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
>          +- Range (0, 100, step=1, splits=Some(8))
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message