spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jacek-lewandowski <>
Subject [GitHub] spark pull request: Spark 3883: SSL support for HttpServer and Akk...
Date Fri, 19 Dec 2014 11:34:57 GMT
Github user jacek-lewandowski commented on the pull request:
    Ok, I think I've found a solution, which is somehow based to what you've shown me. I've
pushed some changes for you to take a look before I continue. 
    The idea for `CoarseGrainedExecutorBackend` is to use SSL settings from "executor" namespace,
defined in worker's properties file - I mean, these settings will be inherited from Spark
worker configuration. They will be used by `CoarseGrainedExecutorBackend` in order to connect
to the driver and fetch the application configuration. Once the application configuration
is fetched, the settings inherited from the worker will be used as defaults and can be overridden
by the application settings.
    Similar approach for `DriverWrapper` (which is used when you run the application in `cluster`
mode). There are two differences - we use "driver" namespace for SSL configuration, and the
way how the configuration options are applied. However the final result is similar.
    Initially, I wanted to use "daemon" settings as defaults ("daemon" namespace is used to
configure SSL for all daemon processes, like Master and Worker). However, it could introduce
some vulnerability because the application could access keystores of those daemon processes.

    So, in short - the administration will have to configure SSL in at least two namespaces
on each node: daemon and executor. It is also recommended to configure it in "driver" namespace
to provide good defaults. In the environment, where authentication is not required, all the
namespaces can be configured with the same settings. 
    What do you think @vanzin ?

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message