Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Fri, 1 Feb 2013 19:58:12 +0000 (UTC)
From: "Michael Allen (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12628553.1358777758049.231186.1359748692980@arcas>
In-Reply-To: <JIRA.12628553.1358777758049@arcas>
References: <JIRA.12628553.1358777758049@arcas>
Subject: [jira] [Commented] (ACCUMULO-980) support pluggable codecs for
 RFile
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/ACCUMULO-980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569023#comment-13569023 ] 

Michael Allen commented on ACCUMULO-980:
----------------------------------------

Hi Keith.  Your comments are spot on in terms of modifying the BCFile vs. RFile classes.  My prototype changes have all been to BCFile thus far, and I think they will be contained to that.  When there's no encryption, then yes, there will be no IV.  

The current design I have in mind should apply equally to anything needing encryption-at-rest within Accumulo, including the WAL, RFiles, and WAL-Map files.  Rather than making one encrypting implementation that fits within the compression codec interface, and another that is then used by the WAL file streaming, we have just one spot where one can easily (and generically based on configuration) obtain an enciphering output stream.

We need to think more about the block caches and their encryption.  In the past crypto systems I've worked on (PGP), we generally only worried about very sensitive pieces of information like keys and passwords being swapped to disk, and not so much about things that were encrypted at rest being realized unencrypted into memory.  Since we have a lot of control here though (versus where PGP sat), we could consider re-encrypting the cached blocks to a key that only exists in memory and for the lifetime of the process.  That way, any swapped blocks would be stored encrypted on the file system to that key, and we don't have to do a lot of key management because they key's really only good for the lifetime of the process.  There's some performance considerations there as well, as cached reads will now require a decrypt.
                
> support pluggable codecs for RFile
> ----------------------------------
>
>                 Key: ACCUMULO-980
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-980
>             Project: Accumulo
>          Issue Type: Improvement
>            Reporter: Adam Fuchs
>            Assignee: Adam Fuchs
>             Fix For: 1.6.0
>
>         Attachments: RFile-Changes-Proposal-V1.pdf
>
>
> As part of the encryption at rest story, RFile should support pluggable modules where it currently has hardcoded options for compression codecs. This is a natural place to add encryption capabilities, as the cost of encryption would likely not be significantly different from the cost of compression, and the block-level integration should maintain the same seek and scan performance. Given the many implementation options for both encryption and compression, it makes sense to have a plugin structure here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira