cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Brown (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-9633) Add ability to encrypt sstables
Date Tue, 19 Apr 2016 20:09:25 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247797#comment-15247797
] 

Jason Brown edited comment on CASSANDRA-9633 at 4/19/16 8:09 PM:
-----------------------------------------------------------------

OK, so here we go with the updates [~bdeggleston] requested.

||9633||
|[branch|https://github.com/jasobrown/cassandra/tree/9633]|
|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-9633-dtest/]|
|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-9633-testall/]|

To encrypt the (primary) index and summary sstable files, in addition to the data file, different
code paths were required.

- As the summary is the once-shot read and different from all other code paths, I made custom
classes for handling it, {{EncryptedSummaryWritableByteChannel}} and {{EncryptedSummaryInputStream}}.
As the summary is intimately linked with the owning sstable & data file, the summary will
simply inherit the key_alias/algo from the sstable, but then has it's IV written to the front
of the summary file.

- The encrypted primary index needs to have it's own 'chunks' file, a la the {{Component.COMPRESSION_INFO}},
so I created {{Component.INDEX_COMPRESSION_INFO}} so that it gets the proper treatment. Thus,
we can use {{CompressedSequentialWriter}} for writing out the index file's offsets, just like
what we do for the compressed data file.

- As with the first version of this patch, encrypting the data file (and now the primary index)
is handled by {{EncryptingCompressor}}. 

WRT to CQL changes, we simply do {{... WITH ENCRYPTION='true'}} to enable the sstable encryption.
All the encryption parameters are already in the yaml, so no need to pass those in separately.
Further, to disable the sstable encryption, simple execute {{ALTER TABLE ... WITH ENCRYPTION='false'}}.
As a side effect of piggy-backing on the compression infrastructure, though, when executing
{{DESCRIBE TABLE}} in cqlsh the encryption params show up as 'compression' data, not as encryption.
I believe all the code for handling the cqlsh describe queries is largely in the python driver,
afaict. 


Some miscellaneous changes:
- {{ICompressor}} got some additional functions for instance-specific values as we need to
carry a unique IV for each cipher.
- {{CipherFactory}} needed a small cleanup wrt caching instances (we were creating a crap
tonne of instances on the fly)
- Apparently I messed up a small part of the merge for #11040, and thus adding it in here
({{HintsService}}). Without this, hints don't get encrypted when enabled.
- added some {{@SupressWarnings}} annotations



was (Author: jasobrown):
OK, so here we go with the updates [~bdeggleston] requested.

||9633||
|[branch|https://github.com/jasobrown/cassandra/tree/9633]|
|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-9633-dtest/]|
|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-9633-testall/]|

To encrypt the (primary) index and summary sstable files, in addition to the data file, different
code paths were required.

-As the summary is the once-shot read and different from all other code paths, I made custom
classes for handling it, {{EncryptedSummaryWritableByteChannel}} and {{EncryptedSummaryInputStream}}.
As the summary is intimately linked with the owning sstable & data file, the summary will
simply inherit the key_alias/algo from the sstable, but then has it's IV written to the front
of the summary file.

- The encrypted primary index needs to have it's own 'chunks' file, a la the {{Component.COMPRESSION_INFO}},
so I created {{Component.INDEX_COMPRESSION_INFO}} so that it gets the proper treatment. Thus,
we can use {{CompressedSequentialWriter}} for writing out the index file's offsets, just like
what we do for the compressed data file.

- As with the first version of this patch, encrypting the data file (and now the primary index)
is handled by {{EncryptingCompressor}}. 

WRT to CQL changes, we simply do {{... WITH ENCRYPTION='true'}} to enable the sstable encryption.
All the encryption parameters are already in the yaml, so no need to pass those in separately.
Further, to disable the sstable encryption, simple execute {{ALTER TABLE ... WITH ENCRYPTION='false'}}.
As a side effect of piggy-backing on the compression infrastructure, though, when executing
{{DESCRIBE TABLE}} in cqlsh the encryption params show up as 'compression' data, not as encryption.
I believe all the code for handling the cqlsh describe queries is largely in the python driver,
afaict. 


Some miscellaneous changes:
- {{ICompressor}} got some additional functions for instance-specific values as we need to
carry a unique IV for each cipher.
- {{CipherFactory}} needed a small cleanup wrt caching instances (we were creating a crap
tonne of instances on the fly)
- Apparently I messed up a small part of the merge for #11040, and thus adding it in here
({{HintsService}}). Without this, hints don't get encrypted when enabled.
- added some {{@SupressWarnings}} annotations


> Add ability to encrypt sstables
> -------------------------------
>
>                 Key: CASSANDRA-9633
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9633
>             Project: Cassandra
>          Issue Type: New Feature
>            Reporter: Jason Brown
>            Assignee: Jason Brown
>              Labels: encryption, security, sstable
>             Fix For: 3.x
>
>
> Add option to allow encrypting of sstables.
> I have a version of this functionality built on cassandra 2.0 that piggy-backs on the
existing sstable compression functionality and ICompressor interface (similar in nature to
what DataStax Enterprise does). However, if we're adding the feature to the main OSS product,
I'm not sure if we want to use the pluggable compression framework or if it's worth investigating
a different path. I think there's a lot of upside in reusing the sstable compression scheme,
but perhaps add a new component in cqlsh for table encryption and a corresponding field in
CFMD.
> Encryption configuration in the yaml can use the same mechanism as CASSANDRA-6018 (which
is currently pending internal review).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message