apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Pujare <san...@datatorrent.com>
Subject Enhancement to support custom SSL configuration
Date Fri, 21 Apr 2017 07:00:15 GMT
Currently StrAM supports only the default Hadoop SSL configuration because
it uses org.apache.hadoop.yarn.webapp.WebApps helper class which has the
limitation of only using the default Hadoop SSL config that is read from
Hadoop's ssl-server.xml resource file. Some users have run into a situation
where Hadoops' SSL keystore is not available on most cluster nodes or the
Stram process doesn't have read access to the keystore even when present.
So there is a need for the Stram to use a custom SSL keystore and
configuration that does not suffer from these limitations.

I am planning to fix this by first fixing WebApps in Hadoop and then
enhancing Stram to use this new fix in Hadoop. I have already submitted a
PR https://github.com/apache/hadoop/pull/213 to Hadoop and one of the the
Hadoop distributors has agreed to accept this fix so I expect it to be
merged very soon.

After that I will enhance Stram to accept the location of a custom
ssl-server.xml file (supplied by the client via a DAG attribute or
property) and use the values from that file to set up the config object to
be passed to WebApps which will end up using the custom SSL configuration.
I have already verified this approach in a prototype.

We will also enhance the Apex client/launcher to distribute the custom SSL
files (XML and the keystore) along with the application jars/resources so
the user does not need to pre-distribute the custom SSL files.

Please let me know your comments.

Sanjay

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message