ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: TDE Implementation details.
Date Wed, 12 Sep 2018 03:41:34 GMT
Nikolay,

Could you please list the limitations of Phase 1? I'm curious what won't be
supported in 2.7.

--
Denis

On Tue, Sep 11, 2018 at 4:29 PM Nikolay Izhikov <nizhikov@apache.org> wrote:

> > As I understand the plan is to get TDE Phase 1 released in 2.7, right?
>
> Yes. We will release TDE in 2.7
>
> > Could you also list what needs to be done at Phase 2 and how much time
> it might take.
>
> Yes, I think Dmitry Ryabov will send Phase 2 design
>
>
> В Вт, 11/09/2018 в 23:27 +0300, Nikolay Izhikov пишет:
> > Hello, Denis.
> >
> > Yes, Vladimir made 2 rounds of review.
> > I planning to fix all issues with implementation in a few days.
> >
> >
> > В Вт, 11/09/2018 в 10:40 -0400, Denis Magda пишет:
> > > Hi Nikolay,
> > >
> > > Has anybody started looking into your request? As I understand the
> plan is
> > > to get TDE Phase 1 released in 2.7, right?
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > >
> > > Could you also list what needs to be done at Phase 2 and how much time
> it
> > > might take.
> > >
> > > --
> > > Denis
> > >
> > > On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <nizhikov@apache.org>
> wrote:
> > >
> > > > Hello, Igniters.
> > > >
> > > > I want to share with you TDE implementation details.
> > > > I think it can simplify review and acception of TDE implementation.
> > > > This mail is copy of wiki page [1].
> > > >
> > > > Please, share your thoughts.
> > > >
> > > > TDE is ready for review [2].
> > > > Please, let me know, who is able to make final review.
> > > >
> > > > This document describes some internal details of TDE.Phase 1
> > > > implementation [3].
> > > > I suggest that reader familiar with the general design described in
> IEP-18
> > > > [4].
> > > >
> > > >
> > > > Cache group key management and node join enhancements:
> > > >
> > > >         1. GridEncryptionManager encapsulates all logic related to
> key
> > > > management:
> > > >                 a. All group encryption keys are stored in MetaStore.
> > > >
> > > >                 b. Joining node sends to cluster:
> > > >                         * Master key digest.
> > > >                         * All group keys saved locally (Map<Integer,
> > > > byte[]>). Keys send over a network in encrypted form.
> > > >
> > > >                 c. Coordinator on new node join:
> > > >                         * Compares new node master key digest with
> the
> > > > local master key digest.
> > > >                         If differs then new node join is rejected.
> > > >
> > > >                         * Compares local and received group keys.
> > > >                         If some key differs then new node join is
> > > > rejected.
> > > >
> > > >                 d. All server nodes:
> > > >                         * If some of received keys are new then they
> save
> > > > locally.
> > > >
> > > >         2. Dynamic cache creation:
> > > >                 a. On server node - Encryption key is generated and
> added
> > > > to DynamicCacheChangeRequest.
> > > >
> > > >                 b. On client node:
> > > >                         * Prior to creation of
> DynamicCacheChangeRequest
> > > > we have to get new encryption key from some server node.
> > > >                         * New request added to solve this -
> > > > GenerateEncryptionKeyRequest.
> > > >
> > > >
> > > > WAL Record encryption:
> > > >
> > > >
> > > >         1. New type of WAL record created – EncryptedRecord.
> > > >
> > > >         2. EncryptedRecord is a container that stores some
> > > > WalRecordCacheGroupAware in encrypted form inside.
> > > >
> > > >         3. Write:
> > > >         Each record for an encrypted group that implements
> > > > WalRecordCacheGroupAware written to WAL in encrypted form.
> > > >         Instead of that record we write EncryptedRecord with plain
> record
> > > > inside(PageSnapshot, PageDeltaRecord, etc).
> > > >
> > > >         4. Read: There are 3 different cases on EncryptedRecord read:
> > > >                 a. WAL restore – we read EncryptedRecord, decrypt
> internal
> > > > record and return it.
> > > >
> > > >                 b. Offline WAL
> iteration(StandaloneWalRecordsIterator) -
> > > > EncryptionSpi is null so wecan’t decrypt EncryptedRecord and just
> return it
> > > > from an iterator.
> > > >
> > > >                 c. Meta storage restore process – On node start we
> restore
> > > > MetaStore.
> > > >                 When we do it no encryption keys are available,
> because
> > > > they are stored in MetaStore.
> > > >                 So we can’t decrypt EncryptedRecord and just return
> it
> > > > from an iterator.
> > > >                 We don't need decrypted record on this step to
> operate
> > > > properly.
> > > >
> > > >
> > > > Page encryption:
> > > >
> > > >
> > > >         1. We have to write on disk pages aligned on 2^n (2kb, 4kb,
> etc)
> > > > for gain maximum perfrormance.
> > > >
> > > >         2. There is a 16 byte overhead for and AES CBC mode.
> > > >
> > > >         3. So we have to preserve 16 bytes in page in memory to get
> > > > encrypted page size equal to 2^n when written it to disk.
> > > >
> > > >         4. PageIO has many methods with pageSize parameter.
> > > >         So for encrypted cache groups page size is calculated as
> > > > cfg.getPageSize() - 16 byte.
> > > >
> > > >
> > > > [1]
> > > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > > > [2] https://github.com/apache/ignite/pull/4167
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-8485
> > > > [4]
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-18%3A+Transparent+Data+Encryption

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message