kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsha <ka...@harsha.io>
Subject Re: Fwd: [DISCUSS] KIP-492 Add java security providers in Kafka Security config
Date Thu, 25 Jul 2019 17:37:05 GMT
Thanks Rajini .

> 4) The main difference between SSL and SASL is that for SSL, you register a
> provider with your own algorithm name and you specify your algorithm name
> in a separate config. This algorithm name can be anything you choose. For
> SASL, we register providers for standard SASL mechanism names. If this KIP
> wants to address the SASL case, then basically you need to add a provider
> ahead of the built-in provider for that standard mechanism name. See below:

We can keep this to SSL only. Given for SASL users can configure a provider for LoginModule.
Unless there is a usecase where we benefit from having a provider config for SASL.



On Thu, Jul 25, 2019, at 5:25 AM, Rajini Sivaram wrote:
> Hi Sandeep/Harsha,
> 
> I don't have any major concerns about this KIP since it solves a specific
> issue and is a relatively minor change. I am unconvinced about the SASL
> case, but it probably is better to add as a config that can be used with
> SASL as well in future anyway.
> 
> Just to complete the conversation above:
> 
> 4) The main difference between SSL and SASL is that for SSL, you register a
> provider with your own algorithm name and you specify your algorithm name
> in a separate config. This algorithm name can be anything you choose. For
> SASL, we register providers for standard SASL mechanism names. If this KIP
> wants to address the SASL case, then basically you need to add a provider
> ahead of the built-in provider for that standard mechanism name. See below:
> 
> 6) JVM starts off with an ordered list of providers, the `java.security`
> file you quote above shows the ordering. When you dynamically add
> providers, you have the choice of adding it anywhere in that list.
> Particularly with SASL where you may have multiple providers with the same
> name, this ordering is significant. Security.insertProviderAt(Provider
> provider, int position) allows you to choose the position. I think the KIP
> intends to add provider at the beginning using Security.addProvider(Provider
> provider), which is basically inserting at position 0. We should clarify
> the behaviour in the KIP.
> 
> We should also clarify how this config will co-exist with existing dynamic
> updates of security providers in the SASL login modules in Kafka. Or
> clarify that you can't override those. What we don't want is
> non-deterministic behaviour which is the biggest issue with these static
> configs.
> 
> 
> On Wed, Jul 24, 2019 at 5:50 PM Harsha <kafka@harsha.io> wrote:
> 
> > Thanks for the details.
> > Rajini, Can you please take a look and let us know if these addresses your
> > concerns.
> >
> > Thanks,
> > Harsha
> >
> > On Mon, Jul 22, 2019, at 9:36 AM, Sandeep Mopuri wrote:
> > > Hi Rajini,
> > >              Thanks for raising the above questions. Please find the
> > > replies below
> > >
> > > On Wed, Jul 17, 2019 at 2:49 AM Rajini Sivaram <rajinisivaram@gmail.com>
> > > wrote:
> > >
> > > > Hi Sandeep,
> > > >
> > > > Thanks for the KIP. A few questions below:
> > > >
> > > >    1. Is the main use case for this KIP adding security providers for
> > SSL?
> > > >    If so, wouldn't a more generic solution like KIP-383 work for this?
> > > >
> > >        We’re trying to solve this for both SSL and SASL. KIP-383 allows
> > the
> > > creation of custom SSLFactory implementation, however adding the provides
> > > to new security algorithms doesn’t involve any new implementation of
> > > SSLFactory. Even after the KIP 383, people still are finding a need for
> > > loading custom keymanager and trustmanager implementations (KIP 486)
> > >
> > >    2. Presumably this config would also apply to clients. If so, have we
> > > >    thought through the implications of changing static JVM-wide
> > security
> > > >    providers in the client applications?
> > > >
> > >        Yes, this config will be applied to clients as well and the
> > > responsibility to face the consequences of adding the security providers
> > > need to be taken by the clients. In cases of resource manager running
> > > streaming applications such as Yarn, Mesos etc.. each user needs to make
> > > sure they are passing these JVM arguments.
> > >
> > >    3. Since client applications can programmatically invoke the Java
> > > >    Security API anyway, isn't the system property described in
> > `Rejected
> > > >    Alternatives` a reasonable solution for brokers?
> > > >
> > >       This is true in a kafka only environment, but with an eco-system of
> > > streaming applications like flink, spark etc which might produce to
> > kafka,
> > > it’s difficult to make changes to all the clients
> > >
> > >    4. We have SASL login modules in Kafka that automatically add security
> > > >    providers for SASL mechanisms not supported by the JVM. We should
> > > > describe
> > > >    the impact of this KIP on those and whether we would now require a
> > > > config
> > > >    to enable these security providers
> > >
> > > In a single JVM, one can register multiple security.providers. By default
> > > JVM itself provides multiple providers and these will not stepped over
> > each
> > > other. The only way to activate a provider is through their registered
> > names
> > > Example:
> > >
> > > $ cat /usr/lib/jvm/jdk-8-oracle-x64/jre/lib/security/java.security
> > > ...
> > > security.provider.1=sun.security.provider.Sun
> > > security.provider.2=sun.security.rsa.SunRsaSign
> > > security.provider.3=sun.security.ec.SunEC
> > > security.provider.4=com.sun.net.ssl.internal.ssl.Provider
> > > security.provider.5=com.sun.crypto.provider.SunJCE
> > > security.provider.6=sun.security.jgss.SunProvider
> > > security.provider.7=com.sun.security.sasl.Provider
> > > security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
> > > security.provider.9=sun.security.smartcardio.SunPCSC
> > > ...
> > >
> > >            A user of a provider will refer through their registered
> > provider
> > >
> > >
> > https://github.com/spiffe/spiffe-example/blob/master/java-spiffe/spiffe-security-provider/src/main/java/spiffe/api/provider/SpiffeProvider.java#L31
> > >
> > >            In the above example , we can register the SpiffeProvider and
> > > multiple other providers into the JVM. When a client or a broker wants to
> > > integrate with SpiffeProvider they need to add the config
> > > ssl.keymanager.alogirhtm = "Spiffe" . Another client can refer to a
> > > different provider or use a default one in the same JVM.
> > >
> > >
> > > >    5. We have been moving away from JVM-wide configs like the default
> > JAAS
> > > >    config since they are hard to test reliably or update dynamically.
> > The
> > > >    replacement config `sasl.jaas.config` doesn't insert a JVM-wide
> > > >    configuration. Have we investigated similar options for the specific
> > > >    scenario we are addressing here?
> > > >
> > >        Yes, that is the case with jaas config, however in the case of
> > > security providers, along with adding the security providers to JVM
> > > properties, one also need to configure the provider algorithm. For
> > example,
> > > in the case of SSL configuration, besides adding the security provider to
> > > the JVM, we need to configure the “ssl.trustmanager.algorithm” and
> > > “ssl.keymanager.algorithm” inorder for the provider implementation to
> > > apply. Different components can opt for different key and trustmanager
> > > algorithms and can work independently simultaneously in the same JVM.
> > This
> > > case is different from the jaas config.
> > >
> > >
> > > >    6. Are we always going to insert new providers at the start of the
> > > >    provider list?
> > >
> > >        Can you please explain what exactly do you mean by this
> > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > > >
> > > >
> > > > On Wed, Jul 17, 2019 at 5:05 AM Harsha <kafka@harsha.io> wrote:
> > > >
> > > > > Thanks for the KIP Sandeep. LGTM.
> > > > >
> > > > > Mani & Rajini, can you please look at the KIP as well.
> > > > >
> > > > > Thanks,
> > > > > Harsha
> > > > >
> > > > > On Tue, Jul 16, 2019, at 2:54 PM, Sandeep Mopuri wrote:
> > > > > > Thanks for the suggestions, made changes accordingly.
> > > > > >
> > > > > > On Tue, Jul 16, 2019 at 9:27 AM Satish Duggana <
> > > > satish.duggana@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Sandeep,
> > > > > > > Thanks for the KIP, I have few comments below.
> > > > > > >
> > > > > > > >>“To take advantage of these custom algorithms,
we want to
> > support
> > > > > java
> > > > > > > security provider parameter in security config. This param
can be
> > > > used
> > > > > by
> > > > > > > kafka brokers or kafka clients(when connecting to the kafka
> > brokers).
> > > > > The
> > > > > > > security providers can also be used for configuring security
> > > > > algorithms in
> > > > > > > SASL based communication.”
> > > > > > >
> > > > > > > You may want to mention use case like
> > > > > > > spiffe.provider.SpiffeProvider[1] in streaming applications
like
> > > > > > > Flink, Spark or Storm etc.
> > > > > > >
> > > > > > > >>"We add new config parameter in KafkaConfig named
> > > > > > > “security.provider.class”. The value of “security.provider”
is
> > > > > expected to
> > > > > > > be a string representing the provider’s full classname.
This
> > provider
> > > > > class
> > > > > > > will be added to the JVM properties through Security.addProvider
> > api.
> > > > > > > Security class can be used to programmatically add the
provider
> > > > > classes to
> > > > > > > the JVM."
> > > > > > >
> > > > > > > It is good to have this property as a list of providers
instead
> > of a
> > > > > > > single property. This will allow configuring multiple providers
> > if it
> > > > > > > is needed in the future without introducing hacky solutions
like
> > > > > > > security.provider.class.name.x, where x is a sequence number.
> > You
> > > > can
> > > > > > > change the property name to “security.provider.class.names”
and
> > its
> > > > > > > value is a list of fully qualified provider class names
> > separated by
> > > > > > > ‘,'.
> > > > > > > For example:
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > security.provider.class.names=spiffe.provider.SpiffeProvider,com.foo.MyProvider
> > > > > > >
> > > > > > > Typo in existing properties section:
> > > > > > > “ssl.provider” instead of “ssl.providers”.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Satish.
> > > > > > >
> > > > > > > 1. https://github.com/spiffe/java-spiffe
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Jul 15, 2019 at 11:41 AM Sandeep Mopuri <
> > mprsai@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > Hello all,
> > > > > > > >
> > > > > > > > I'd like to start a discussion thread for KIP-492.
> > > > > > > > This KIP plans on introducing a new security config
parameter
> > for a
> > > > > > > custom
> > > > > > > > security providers. Please take a look and let me
know what do
> > you
> > > > > think.
> > > > > > > >
> > > > > > > > More information can be found here:
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-492%3A+Add+java+security+providers+in+Kafka+Security+config
> > > > > > > > --
> > > > > > > > Thanks,
> > > > > > > > Sai Sandeep
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > M.Sai Sandeep
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Thanks,
> > > M.Sai Sandeep
> > >
> > >
> > > --
> > > Thanks,
> > > M.Sai Sandeep
> > >
> >
>

Mime
View raw message