ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: TDE Implementation details.
Date Tue, 11 Sep 2018 14:40:46 GMT
Hi Nikolay,

Has anybody started looking into your request? As I understand the plan is
to get TDE Phase 1 released in 2.7, right?
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473

Could you also list what needs to be done at Phase 2 and how much time it
might take.

--
Denis

On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <nizhikov@apache.org> wrote:

> Hello, Igniters.
>
> I want to share with you TDE implementation details.
> I think it can simplify review and acception of TDE implementation.
> This mail is copy of wiki page [1].
>
> Please, share your thoughts.
>
> TDE is ready for review [2].
> Please, let me know, who is able to make final review.
>
> This document describes some internal details of TDE.Phase 1
> implementation [3].
> I suggest that reader familiar with the general design described in IEP-18
> [4].
>
>
> Cache group key management and node join enhancements:
>
>         1. GridEncryptionManager encapsulates all logic related to key
> management:
>                 a. All group encryption keys are stored in MetaStore.
>
>                 b. Joining node sends to cluster:
>                         * Master key digest.
>                         * All group keys saved locally (Map<Integer,
> byte[]>). Keys send over a network in encrypted form.
>
>                 c. Coordinator on new node join:
>                         * Compares new node master key digest with the
> local master key digest.
>                         If differs then new node join is rejected.
>
>                         * Compares local and received group keys.
>                         If some key differs then new node join is
> rejected.
>
>                 d. All server nodes:
>                         * If some of received keys are new then they save
> locally.
>
>         2. Dynamic cache creation:
>                 a. On server node - Encryption key is generated and added
> to DynamicCacheChangeRequest.
>
>                 b. On client node:
>                         * Prior to creation of DynamicCacheChangeRequest
> we have to get new encryption key from some server node.
>                         * New request added to solve this -
> GenerateEncryptionKeyRequest.
>
>
> WAL Record encryption:
>
>
>         1. New type of WAL record created – EncryptedRecord.
>
>         2. EncryptedRecord is a container that stores some
> WalRecordCacheGroupAware in encrypted form inside.
>
>         3. Write:
>         Each record for an encrypted group that implements
> WalRecordCacheGroupAware written to WAL in encrypted form.
>         Instead of that record we write EncryptedRecord with plain record
> inside(PageSnapshot, PageDeltaRecord, etc).
>
>         4. Read: There are 3 different cases on EncryptedRecord read:
>                 a. WAL restore – we read EncryptedRecord, decrypt internal
> record and return it.
>
>                 b. Offline WAL iteration(StandaloneWalRecordsIterator) -
> EncryptionSpi is null so wecan’t decrypt EncryptedRecord and just return it
> from an iterator.
>
>                 c. Meta storage restore process – On node start we restore
> MetaStore.
>                 When we do it no encryption keys are available, because
> they are stored in MetaStore.
>                 So we can’t decrypt EncryptedRecord and just return it
> from an iterator.
>                 We don't need decrypted record on this step to operate
> properly.
>
>
> Page encryption:
>
>
>         1. We have to write on disk pages aligned on 2^n (2kb, 4kb, etc)
> for gain maximum perfrormance.
>
>         2. There is a 16 byte overhead for and AES CBC mode.
>
>         3. So we have to preserve 16 bytes in page in memory to get
> encrypted page size equal to 2^n when written it to disk.
>
>         4. PageIO has many methods with pageSize parameter.
>         So for encrypted cache groups page size is calculated as
> cfg.getPageSize() - 16 byte.
>
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> [2] https://github.com/apache/ignite/pull/4167
> [3] https://issues.apache.org/jira/browse/IGNITE-8485
> [4]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-18%3A+Transparent+Data+Encryption
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message