ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolay Izhikov <nizhi...@apache.org>
Subject Re: TDE Implementation details.
Date Tue, 11 Sep 2018 20:29:01 GMT
> As I understand the plan is to get TDE Phase 1 released in 2.7, right?

Yes. We will release TDE in 2.7

> Could you also list what needs to be done at Phase 2 and how much time it might take.

Yes, I think Dmitry Ryabov will send Phase 2 design 


В Вт, 11/09/2018 в 23:27 +0300, Nikolay Izhikov пишет:
> Hello, Denis.
> 
> Yes, Vladimir made 2 rounds of review.
> I planning to fix all issues with implementation in a few days.
> 
> 
> В Вт, 11/09/2018 в 10:40 -0400, Denis Magda пишет:
> > Hi Nikolay,
> > 
> > Has anybody started looking into your request? As I understand the plan is
> > to get TDE Phase 1 released in 2.7, right?
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > 
> > Could you also list what needs to be done at Phase 2 and how much time it
> > might take.
> > 
> > --
> > Denis
> > 
> > On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <nizhikov@apache.org> wrote:
> > 
> > > Hello, Igniters.
> > > 
> > > I want to share with you TDE implementation details.
> > > I think it can simplify review and acception of TDE implementation.
> > > This mail is copy of wiki page [1].
> > > 
> > > Please, share your thoughts.
> > > 
> > > TDE is ready for review [2].
> > > Please, let me know, who is able to make final review.
> > > 
> > > This document describes some internal details of TDE.Phase 1
> > > implementation [3].
> > > I suggest that reader familiar with the general design described in IEP-18
> > > [4].
> > > 
> > > 
> > > Cache group key management and node join enhancements:
> > > 
> > >         1. GridEncryptionManager encapsulates all logic related to key
> > > management:
> > >                 a. All group encryption keys are stored in MetaStore.
> > > 
> > >                 b. Joining node sends to cluster:
> > >                         * Master key digest.
> > >                         * All group keys saved locally (Map<Integer,
> > > byte[]>). Keys send over a network in encrypted form.
> > > 
> > >                 c. Coordinator on new node join:
> > >                         * Compares new node master key digest with the
> > > local master key digest.
> > >                         If differs then new node join is rejected.
> > > 
> > >                         * Compares local and received group keys.
> > >                         If some key differs then new node join is
> > > rejected.
> > > 
> > >                 d. All server nodes:
> > >                         * If some of received keys are new then they save
> > > locally.
> > > 
> > >         2. Dynamic cache creation:
> > >                 a. On server node - Encryption key is generated and added
> > > to DynamicCacheChangeRequest.
> > > 
> > >                 b. On client node:
> > >                         * Prior to creation of DynamicCacheChangeRequest
> > > we have to get new encryption key from some server node.
> > >                         * New request added to solve this -
> > > GenerateEncryptionKeyRequest.
> > > 
> > > 
> > > WAL Record encryption:
> > > 
> > > 
> > >         1. New type of WAL record created – EncryptedRecord.
> > > 
> > >         2. EncryptedRecord is a container that stores some
> > > WalRecordCacheGroupAware in encrypted form inside.
> > > 
> > >         3. Write:
> > >         Each record for an encrypted group that implements
> > > WalRecordCacheGroupAware written to WAL in encrypted form.
> > >         Instead of that record we write EncryptedRecord with plain record
> > > inside(PageSnapshot, PageDeltaRecord, etc).
> > > 
> > >         4. Read: There are 3 different cases on EncryptedRecord read:
> > >                 a. WAL restore – we read EncryptedRecord, decrypt internal
> > > record and return it.
> > > 
> > >                 b. Offline WAL iteration(StandaloneWalRecordsIterator) -
> > > EncryptionSpi is null so wecan’t decrypt EncryptedRecord and just return
it
> > > from an iterator.
> > > 
> > >                 c. Meta storage restore process – On node start we restore
> > > MetaStore.
> > >                 When we do it no encryption keys are available, because
> > > they are stored in MetaStore.
> > >                 So we can’t decrypt EncryptedRecord and just return it
> > > from an iterator.
> > >                 We don't need decrypted record on this step to operate
> > > properly.
> > > 
> > > 
> > > Page encryption:
> > > 
> > > 
> > >         1. We have to write on disk pages aligned on 2^n (2kb, 4kb, etc)
> > > for gain maximum perfrormance.
> > > 
> > >         2. There is a 16 byte overhead for and AES CBC mode.
> > > 
> > >         3. So we have to preserve 16 bytes in page in memory to get
> > > encrypted page size equal to 2^n when written it to disk.
> > > 
> > >         4. PageIO has many methods with pageSize parameter.
> > >         So for encrypted cache groups page size is calculated as
> > > cfg.getPageSize() - 16 byte.
> > > 
> > > 
> > > [1]
> > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > > [2] https://github.com/apache/ignite/pull/4167
> > > [3] https://issues.apache.org/jira/browse/IGNITE-8485
> > > [4]
> > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-18%3A+Transparent+Data+Encryption
Mime
View raw message