accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Allen (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-980) support pluggable codecs for RFile
Date Fri, 01 Feb 2013 19:58:12 GMT


Michael Allen commented on ACCUMULO-980:

Hi Keith.  Your comments are spot on in terms of modifying the BCFile vs. RFile classes. 
My prototype changes have all been to BCFile thus far, and I think they will be contained
to that.  When there's no encryption, then yes, there will be no IV.  

The current design I have in mind should apply equally to anything needing encryption-at-rest
within Accumulo, including the WAL, RFiles, and WAL-Map files.  Rather than making one encrypting
implementation that fits within the compression codec interface, and another that is then
used by the WAL file streaming, we have just one spot where one can easily (and generically
based on configuration) obtain an enciphering output stream.

We need to think more about the block caches and their encryption.  In the past crypto systems
I've worked on (PGP), we generally only worried about very sensitive pieces of information
like keys and passwords being swapped to disk, and not so much about things that were encrypted
at rest being realized unencrypted into memory.  Since we have a lot of control here though
(versus where PGP sat), we could consider re-encrypting the cached blocks to a key that only
exists in memory and for the lifetime of the process.  That way, any swapped blocks would
be stored encrypted on the file system to that key, and we don't have to do a lot of key management
because they key's really only good for the lifetime of the process.  There's some performance
considerations there as well, as cached reads will now require a decrypt.
> support pluggable codecs for RFile
> ----------------------------------
>                 Key: ACCUMULO-980
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>            Reporter: Adam Fuchs
>            Assignee: Adam Fuchs
>             Fix For: 1.6.0
>         Attachments: RFile-Changes-Proposal-V1.pdf
> As part of the encryption at rest story, RFile should support pluggable modules where
it currently has hardcoded options for compression codecs. This is a natural place to add
encryption capabilities, as the cost of encryption would likely not be significantly different
from the cost of compression, and the block-level integration should maintain the same seek
and scan performance. Given the many implementation options for both encryption and compression,
it makes sense to have a plugin structure here.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message