spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <>
Subject [jira] [Commented] (SPARK-23850) We should not redact username|user|url from UI by default
Date Tue, 24 Apr 2018 00:13:00 GMT


Marcelo Vanzin commented on SPARK-23850:

Yeah, things like {{java.vendor.url}} end up redacted too...

But there's the issue of JDBC drivers allowing passwords in their URL. Though they generally
allow the password to be provided separately in a properties object too, which is preferable.

Also, it turns out other parts of SQL use a different way of redacting things added in SPARK-22791.
The way I read that, the code added to {{SaveIntoDataSourceCommand}} is now redundant, since
paths that print plans will be automatically redacted by the code in {{QueryExecution}}.

That change added a separate config that defaults to the value of the config in core. If we
change that to be a separate config instead of falling back to the code config, we could have
different defaults (leaving the URL alone on the core side), but that changes behavior slightly.

> We should not redact username|user|url from UI by default
> ---------------------------------------------------------
>                 Key: SPARK-23850
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: Web UI
>    Affects Versions: 2.2.1
>            Reporter: Thomas Graves
>            Priority: Major
> SPARK-22479 was filed to not print the log jdbc credentials, but in there they also
added  the username and url to be redacted.  I'm not sure why these were added and to me
by default these do not have security concerns.  It makes it more usable by default to be
able to see these things.  Users with high security concerns can simply add them in their
> Also on yarn just redacting url doesn't secure anything because if you go to the environment
ui page you see all sorts of paths and really its just confusing that some of its redacted
and other parts aren't.  If this was specifically for jdbc I think it needs to be just applied
there and not broadly.
> If we remove these we need to test what the jdbc driver is going to log from SPARK-22479.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message