storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Parth-Brahmbhatt <>
Subject [GitHub] incubator-storm pull request: STORM-346: added AutoHDFS class that...
Date Fri, 18 Jul 2014 20:14:29 GMT
Github user Parth-Brahmbhatt commented on the pull request:
    Here is what I have so far:
    User can specify as "nimbus.credential.renewers.classes" and AutoHDFS will
only implement ICredentialsRenewer. In the prepare phase of, which should be
called on nimbus startup, we can get the HDFS credentials. However, I don't think the topology
submitter user will be available at that time so we will not be able to get the token on behalf
of the user but only as nimbus which I feel is unacceptable. 
    In order to actually get the credentials as topology submitter user, we either need a
new Interface that will run on nimbus when a topology is submitted as part of submitTopologyWithOpts
implementation or we can add getCredentialForUser(Map conf) method to ICredentialsRenewer
interface and call that as part of submitTopologyWithOpts. I personally prefer not to pollute
the ICredentialsRenewer interface. Let me know if you have better alternatives or prefer one
over another.
    I have one last question. The ICredentialsRenewer implementations seems to be loaded by
reading "nimbus.credential.renewers.classes" config at startup by nimbus. This means the users
who have a running nimbus and wants to use AutoHDFS or any other implementation of ICredentialsRenewer
will have to change the config and restart the nimbus. Is that acceptable? 

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message