apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chinmay Kolhatkar <chin...@datatorrent.com>
Subject Re: Encrypted Streams
Date Tue, 15 Dec 2015 08:58:51 GMT
Chandni,

The reason I proposed this is because, currently we don't have any way to
have the streaming data in encrypted format.
What Tim mentioned is also a good idea and the DAG level attribute can
decide for it.

But I feel its a useful utility for one who wants to keep the data secure
at every stage of processing.

-Chinmay.


~ Chinmay.

On Tue, Dec 15, 2015 at 1:45 PM, Chandni Singh <chandni@datatorrent.com>
wrote:

> The processing of data, which is the entire pipeline, will run usually in a
> secured cluster. The data may need to be encrypted only when it is done
> processing by the DAG and leaves the cluster.
>
>
>
> On Tue, Dec 15, 2015 at 12:10 AM, Timothy Farkas <tim@datatorrent.com>
> wrote:
>
> > I think encryption of data sent across the wire and operator logic are
> > orthogonal. The user should just have to set DAG level attribute to
> > enable/disable encryption, without having to write any encryption related
> > code. I think this would require changes to the Buffer Server publisher
> and
> > subscriber though.
> >
> > On Mon, Dec 14, 2015 at 11:27 PM, Chandni Singh <chandni@datatorrent.com
> >
> > wrote:
> >
> > > When we are dealing with secured data, the usual scenarios are that you
> > get
> > > encrypted data.
> > > This data need to decrypt and then perform other functions on it. The
> > > output of the dag is then encrypted.
> > >
> > > In the past we have solved these use cases by performing
> > > decryption/encryption in the operator.
> > > IMO the operator approach works better because these processes may
> > require
> > > invoking utilities and also operators can be configured easily using
> > > properties.
> > >
> > > Chandni
> > >
> > > On Mon, Dec 14, 2015 at 10:34 PM, Sandesh Hegde <
> sandesh@datatorrent.com
> > >
> > > wrote:
> > >
> > > > Well we have committers from bank, their feedback will be really
> > > valuable.
> > > >
> > > > On Mon, Dec 14, 2015 at 10:30 PM Priyanka Gugale <
> > > priyanka@datatorrent.com
> > > > >
> > > > wrote:
> > > >
> > > > > Sounds good. This is good feature for banks and security domain.
> > > > > One suggestion: We can do key management ourself at application
> (may
> > be
> > > > by
> > > > > providing default keys) and there should be an option to override
> > keys
> > > if
> > > > > user really want to do so.
> > > > >
> > > > > -Priyanka
> > > > >
> > > > > On Tue, Dec 15, 2015 at 11:37 AM, Chinmay Kolhatkar <
> > > > > chinmay@datatorrent.com
> > > > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I wanted to propose an idea using which one can have encrypted
> > stream
> > > > > > flowing in a DAG.
> > > > > >
> > > > > > Basically, the idea is to create a new EncryptedInputPort which
> > will
> > > > > extend
> > > > > > from DefaultInputPort and will return a StreamCodec object which
> > will
> > > > > take
> > > > > > care of encryption/decryption.
> > > > > > As the same StreamCodec object will be used at OutputPort, the
> > > > encryption
> > > > > > can be done in toByteArray method at Output port and decryption
> can
> > > be
> > > > > done
> > > > > > in fromByteArray at Input port.
> > > > > >
> > > > > > By default we can support some basic encryption algorithms like
> RSA
> > > and
> > > > > DSA
> > > > > > where user need to provide the key(s) to EncryptedInputPort.
> > > > > >
> > > > > > Any thoughts?
> > > > > >
> > > > > > ~ Chinmay.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message