ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolay Izhikov <nizhi...@apache.org>
Subject Re: TDE Implementation details.
Date Tue, 11 Sep 2018 20:27:23 GMT
Hello, Denis.

Yes, Vladimir made 2 rounds of review.
I planning to fix all issues with implementation in a few days.


В Вт, 11/09/2018 в 10:40 -0400, Denis Magda пишет:
> Hi Nikolay,
> 
> Has anybody started looking into your request? As I understand the plan is
> to get TDE Phase 1 released in 2.7, right?
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> 
> Could you also list what needs to be done at Phase 2 and how much time it
> might take.
> 
> --
> Denis
> 
> On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <nizhikov@apache.org> wrote:
> 
> > Hello, Igniters.
> > 
> > I want to share with you TDE implementation details.
> > I think it can simplify review and acception of TDE implementation.
> > This mail is copy of wiki page [1].
> > 
> > Please, share your thoughts.
> > 
> > TDE is ready for review [2].
> > Please, let me know, who is able to make final review.
> > 
> > This document describes some internal details of TDE.Phase 1
> > implementation [3].
> > I suggest that reader familiar with the general design described in IEP-18
> > [4].
> > 
> > 
> > Cache group key management and node join enhancements:
> > 
> >         1. GridEncryptionManager encapsulates all logic related to key
> > management:
> >                 a. All group encryption keys are stored in MetaStore.
> > 
> >                 b. Joining node sends to cluster:
> >                         * Master key digest.
> >                         * All group keys saved locally (Map<Integer,
> > byte[]>). Keys send over a network in encrypted form.
> > 
> >                 c. Coordinator on new node join:
> >                         * Compares new node master key digest with the
> > local master key digest.
> >                         If differs then new node join is rejected.
> > 
> >                         * Compares local and received group keys.
> >                         If some key differs then new node join is
> > rejected.
> > 
> >                 d. All server nodes:
> >                         * If some of received keys are new then they save
> > locally.
> > 
> >         2. Dynamic cache creation:
> >                 a. On server node - Encryption key is generated and added
> > to DynamicCacheChangeRequest.
> > 
> >                 b. On client node:
> >                         * Prior to creation of DynamicCacheChangeRequest
> > we have to get new encryption key from some server node.
> >                         * New request added to solve this -
> > GenerateEncryptionKeyRequest.
> > 
> > 
> > WAL Record encryption:
> > 
> > 
> >         1. New type of WAL record created – EncryptedRecord.
> > 
> >         2. EncryptedRecord is a container that stores some
> > WalRecordCacheGroupAware in encrypted form inside.
> > 
> >         3. Write:
> >         Each record for an encrypted group that implements
> > WalRecordCacheGroupAware written to WAL in encrypted form.
> >         Instead of that record we write EncryptedRecord with plain record
> > inside(PageSnapshot, PageDeltaRecord, etc).
> > 
> >         4. Read: There are 3 different cases on EncryptedRecord read:
> >                 a. WAL restore – we read EncryptedRecord, decrypt internal
> > record and return it.
> > 
> >                 b. Offline WAL iteration(StandaloneWalRecordsIterator) -
> > EncryptionSpi is null so wecan’t decrypt EncryptedRecord and just return it
> > from an iterator.
> > 
> >                 c. Meta storage restore process – On node start we restore
> > MetaStore.
> >                 When we do it no encryption keys are available, because
> > they are stored in MetaStore.
> >                 So we can’t decrypt EncryptedRecord and just return it
> > from an iterator.
> >                 We don't need decrypted record on this step to operate
> > properly.
> > 
> > 
> > Page encryption:
> > 
> > 
> >         1. We have to write on disk pages aligned on 2^n (2kb, 4kb, etc)
> > for gain maximum perfrormance.
> > 
> >         2. There is a 16 byte overhead for and AES CBC mode.
> > 
> >         3. So we have to preserve 16 bytes in page in memory to get
> > encrypted page size equal to 2^n when written it to disk.
> > 
> >         4. PageIO has many methods with pageSize parameter.
> >         So for encrypted cache groups page size is calculated as
> > cfg.getPageSize() - 16 byte.
> > 
> > 
> > [1]
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > [2] https://github.com/apache/ignite/pull/4167
> > [3] https://issues.apache.org/jira/browse/IGNITE-8485
> > [4]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-18%3A+Transparent+Data+Encryption
> > 
Mime
View raw message