heron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjeev Kulkarni <sanjee...@gmail.com>
Subject Re: Inter-Container Encryption in Heron
Date Sat, 16 Sep 2017 06:29:40 GMT
Having a separate keypair per container and keeping that in zk has scaling
issues. Assuming 2048 bit encryption which is increasingly becoming
standard, a 100 container topology will have an extremely large pplan
footprint at zk to cause issues.
Secondly, we are talking about encrypting traffic between tmaster and
stmgrs as well.
I want to solicit what would actually be ok vs not-ok from would be users
as well. I believe this was something the Google guys had wanted. Andrew
any inputs?

On Thu, Sep 14, 2017 at 10:15 PM, FatJ Love <huijun.wu.2010@gmail.com>
wrote:

> It is an issue.
> The current Heron also has this issue. Tmaste does not authenticate the
> stmgr regsitration. If two stmgrs regsiter with the same name, tmaster just
> waits for the old one to quit.
>
> For a secure environment, the host may have its key-pair and pub key signed
> by some CA already. We may use the host key/cert instead of generating
> container keys.
>
> On Thu, Sep 14, 2017 at 7:23 PM, Matteo Merli <matteo.merli@gmail.com>
> wrote:
>
> > On Thu, Sep 14, 2017 at 5:26 PM FatJ Love <huijun.wu.2010@gmail.com>
> > wrote:
> >
> > > The pri keys never appear on network.
> > > The pub keys are transfered over network. Pub keys are supposed to be
> > > public. If pub key is changed ,  it cannot match the pri key stored in
> > > container
> > >
> >
> > That's correct but it might allow some other party to connect. E.g.  I
> > submit a modified public key and then I might be able to connect to other
> > bolts and send tuples.
> >
> > Another option is that by spoofing the container IP one can impersonate
> it
> > (with the tampered public key) and receive tuples.
> >
> >
> > >
> > > On Thu, Sep 14, 2017 at 5:17 PM, Matteo Merli <mmerli@apache.org>
> wrote:
> > >
> > > > > 2 then each container sends its (pub) to tmaster and embeds all pub
> > > keys
> > > > (N
> > > > pub keys) in physical plan to all containers.
> > > >
> > > > If this is transmitted over a non-secure channel, wouldn't there be a
> > > > chance to tamper the keys?
> > > >
> > > > On Thu, Sep 14, 2017 at 5:12 PM FatJ Love <huijun.wu.2010@gmail.com>
> > > > wrote:
> > > >
> > > > > For key distribution and management:
> > > > > 1 each container generate a key pair (pri, pub) when the container
> > > > > launches.
> > > > > 2 then each container sends its (pub) to tmaster and embeds all pub
> > > keys
> > > > (N
> > > > > pub keys) in physical plan to all containers.
> > > > > 3 now each container has its own pri key and all pub key (one pri
> key
> > > > and N
> > > > > pub keys)
> > > > >
> > > > > On Thu, Sep 14, 2017 at 5:07 PM, FatJ Love <
> huijun.wu.2010@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Assume there are 2 containers A and B
> > > > > >
> > > > > > --- case 1, only one key (pub, pri) for container A and
> container B
> > > > > > 1. contaienr A sends packet to container B
> > > > > > container A enc packet with key pub
> > > > > > container B dec packet with key pri.
> > > > > > 2. contaienr B sends packet to container A
> > > > > > container B enc packet with key pub
> > > > > > container A dec packet with key pri.
> > > > > > Both A and B have same (pub, pri), why not use symmetric key
?
> > > > > >
> > > > > > --case 2, container A has (pubA, priA). container B has (pubB,
> > priB)
> > > > > > 1 contaienr A sends packet to container B
> > > > > > container A enc packet with key pubB
> > > > > > container B dec packet with key priB.
> > > > > > 2 contaienr B sends packet to container A
> > > > > > container B enc packet with key pubA
> > > > > > container A dec packet with key priA.
> > > > > > The packet is encrypted.
> > > > > >
> > > > > > ---case 3, same assumption with case2
> > > > > > 1 contaienr A sends packet to container B
> > > > > > container A enc packet with key priA
> > > > > > container B dec packet with key pubA.
> > > > > > 2 contaienr B sends packet to container A
> > > > > > container B enc packet with key priB
> > > > > > container A dec packet with key pubB.
> > > > > > container A authenticates B and B can authenticate A as well.
> > > > > >
> > > > > > ---case 4, combine case 2 and case3
> > > > > > both encrytion and authentication
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Sep 14, 2017 at 4:34 PM, Sanjeev Kulkarni <
> > > sanjeevrk@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> At this point, the thought is towards having one key pair
> accross
> > > the
> > > > > >> entire topology. I'm not sure one key pair per container
will
> add
> > > that
> > > > > >> much
> > > > > >> more security. Thoughts?
> > > > > >>
> > > > > >> On Thu, Sep 14, 2017 at 4:30 PM, FatJ Love <
> > > huijun.wu.2010@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > do you plan to use one key-pair for all the contaienrs
or
> > > > > >> > each container has one-key-pair?
> > > > > >> >
> > > > > >> > On Thu, Sep 14, 2017 at 3:39 PM, Sanjeev Kulkarni <
> > > > > sanjeevrk@gmail.com>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > The keys ideally should be different for every
topology. And
> > > > ideally
> > > > > >> they
> > > > > >> > > should be created in a just in time manner while
launching
> the
> > > > > >> topology.
> > > > > >> > > Otherwise, the chance that keys are no longer
secret
> > increases.
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > On Thu, Sep 14, 2017 at 3:34 PM, Karthik Ramasamy
<
> > > > > karthik@streaml.io
> > > > > >> >
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Another option is keep this configuration
in heron cli.
> For
> > > > > example,
> > > > > >> > > >
> > > > > >> > > > heron config <cluster> set --private-key
"...."
> > > > > >> > > > heron config <cluster> set --publick-key
"...."
> > > > > >> > > >
> > > > > >> > > > Once the user sets it, then we can use these
keys?
> > > > > >> > > >
> > > > > >> > > > cheers
> > > > > >> > > > /karthik
> > > > > >> > > >
> > > > > >> > > > On Thu, Sep 14, 2017 at 3:11 PM, Sanjeev
Kulkarni <
> > > > > >> sanjeevrk@gmail.com
> > > > > >> > >
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > Hi all,
> > > > > >> > > > > I believe at-least one current user
of Heron is
> interested
> > > in
> > > > > >> > > encrypting
> > > > > >> > > > > their inter-container data flow within
a Heron topology.
> > > > Since
> > > > > >> > > > > inter-container traffic is between stmgrs,
and because
> > > stmgrs
> > > > > use
> > > > > >> > > > libevent
> > > > > >> > > > > bufferevents for their transport, adding
ssl in
> transport
> > > > layer
> > > > > >> > between
> > > > > >> > > > > stmgrs is fairly straightforward.
> > > > > >> > > > > The bigger question is how credentials
are
> > > > > >> > managed/transferred/stored.
> > > > > >> > > > One
> > > > > >> > > > > approach would to pass the public/private
key as an
> > argument
> > > > to
> > > > > >> the
> > > > > >> > > heron
> > > > > >> > > > > cli while submitting the job. These
will be stored in
> the
> > > > > uploader
> > > > > >> > > > > alongside topology jars and downloaded
by the containers
> > > upon
> > > > > >> start.
> > > > > >> > > The
> > > > > >> > > > > one issue with this approach is that
the keys need to be
> > > > secured
> > > > > >> by
> > > > > >> > the
> > > > > >> > > > > uploader.
> > > > > >> > > > > I believe Kubernetes has provision for
keeping secrets
> for
> > > > jobs,
> > > > > >> but
> > > > > >> > it
> > > > > >> > > > > might not be portable to other scheduler
environments. A
> > way
> > > > > >> around
> > > > > >> > > this
> > > > > >> > > > > would be to create an spi for keeping
secrets in
> > heron/spi,
> > > > but
> > > > > >> not
> > > > > >> > > sure
> > > > > >> > > > > what others feel about this.
> > > > > >> > > > > Any other ideas?
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > > --
> > > > Matteo Merli
> > > > <mmerli@apache.org>
> > > >
> > >
> > --
> > Matteo Merli
> > <mmerli@apache.org>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message