accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Slacum <wsla...@gmail.com>
Subject Re: [DISCUSS] What to do about encryption at rest?
Date Mon, 02 Nov 2015 17:27:09 GMT
Is "the code being 'at rest'" you making a funny about active development?
Making sure I haven't lost my ability to get jokes :)

I see two reasons why the code would be inactive: the feature is good
enough as is or it's not interesting enough to attract attention.
Considering it's not public API, there are no discussions to bring into the
public API, and there's no effort to document how to use it, my intuition
tells me that there isn't enough interest in it from a project perspective.

>From a user perspective, I've been getting asked about it when I work with
Accumulo users. My recommendation, exclusively, is to use HDFS encryption
because I can go to Hadoop's website and find documentation on it. When I
go to find documentation on Accumulo's offerings, any usability information
comes from vendor SlideShares. Most mentions of the feature on official
Apache Accumulo channels echo Christopher's sentiments on the feature being
experimental and not being officially recommended for use.

I wouldn't want to rip out the feature first and then figure things out
later. Sean already alluded to it, but a roadmap should contain something
(tool or documentation) to help users migrate if we go down that route.

What I'm trying to figure out is, when the question of "How do I do
encryption at rest in Accumulo?" comes up, what is our community's answer?

If we went down the route of using HDFS encryption zones, can we offer the
same features? At the very least, we'd be offering the same database-level
encryption scheme. I don't know the details of "more advanced key stores",
but it seems like we could potentially take any custom implementation and
map it to a KeyProvider [1]. I could also envision table level encryption
being implementable via zones, but probably not down to the column family
level.

[1]
https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/crypto/key/KeyProvider.html


On Sun, Nov 1, 2015 at 10:19 AM, Adam Fuchs <afuchs@apache.org> wrote:

> Responses inline.
>
> Adam
>
> On Nov 1, 2015 9:58 AM, "Christopher" <ctubbsii@apache.org> wrote:
> >
> > 1. I'm not sure I'd call an incomplete solution 'great'. What it does is
> > provide partial encryption-at-rest protection (unless you're running
> > without walogs, and have good integration with some external secure key
> > management faculty, and then it's probably fine).
>
> The only thing that doesn't get encrypted is a temporary WAL recovery file.
> That is a project we should take on, but it does not imply that the
> existing features are not valuable. With HDFS encryption options this would
> now be a much easier project to take on. Also, the users I know that use
> encryption at rest do so with a more secure key store than the default.
>
> >
> > 2. I'm concerned that anybody using Accumulo's E-A-R don't necessarily
> > realize its current shortcomings, or its lack of upstream maintenance
> > support (which it has not been receiving). It may be the case that these
> > users have support from an intermediary, and do understand the
> > shortcomings... I don't know, but it's a concern.
>
> Anybody that creates a secure system has to analyze the security of the
> system as a whole. Accumulo's encryption at rest is one part of the
> solution. Taking away the tool without providing an alternative does
> nothing to improve the security of systems built on Accumulo.
>
> >
> > 3. Correction: it has been an explicitly experimental feature and an
> > incomplete one, which hasn't really been touched in two years, and has
> been
> > explicitly excluded by the community for being public API because of its
> > incompleteness. Age doesn't determine public API status. The community
> does.
>
> People are using it, so we have to consider the implications of whatever
> changes we make and weigh against the benefits. I believe the last bug fix
> was done this year, so I would argue it is being maintained. Changes to our
> encryption at rest implementation will have consequences for those users.
> There had better be a clear benefit if we break their systems.
>
> >
> > 4. Has Accumulo's been evaluated for security and performance? By whom?
> Is
> > it published?
>
> Yes, there have been several talks at meetups and conferences that discuss
> the security and performance of the current solution.
>
> >
> > On Sun, Nov 1, 2015, 08:55 Adam Fuchs <afuchs@apache.org> wrote:
> >
> > > There's another way to look at the state of Accumulo's encryption at
> rest:
> > > 1. Encryption at rest works great for what it does, and the code being
> "at
> > > rest" isn't necessarily a problem
> > > 2. Several organizations are using Accumulo's encryption at rest
> > > effectively in operations
> > > 3. Encryption at rest has been a supported configuration option for
> over
> > > two years with established plugin interfaces, and therefore it should
> be
> > > considered part of the public API
> > > 4. Upstream alternatives (to my knowledge) have not been analyzed for
> > > performance or security
> > >
> > > The given option #2 would at least require an analysis of alternatives,
> and
> > > we would have to decide what to do about backwards compatibility for
> users
> > > using custom key stores and encryption strategies that may or may not
> be
> > > supported by upstream alternatives.
> > >
> > > As far as option #1 goes, I can get behind encouraging people to take
> up
> > > projects to improve Accumulo's encryption. I think we're already going
> down
> > > this path, but without having identified resources to do the
> improvements.
> > > Any volunteers?
> > >
> > > Adam
> > >
> > >
> > > On Fri, Oct 30, 2015 at 4:22 PM, William Slacum <wslacum@gmail.com>
> wrote:
> > >
> > > > So I've been looking into options for providing encryption at rest,
> and
> > > it
> > > > seems like what Accumulo has is abandonware from a project
> perspective.
> > > > There is no official documentation on how to perform encryption at
> rest,
> > > > and the best information from its status comes from year (or greater)
> old
> > > > ticket comments about how the feature is still experimental. Recently
> > > there
> > > > was a talk that described using HDFS encryption zones as an
> alternative.
> > > >
> > > > From my perspective, this is what I see as the current situation:
> > > >
> > > > 1- Encryption at rest in Accumulo isn't actively being worked on
> > > > 2- Encryption at rest in Accumulo isn't part of the public API or
> > > marketed
> > > > capabilities
> > > > 3- Documentation for what does exist is scattered throughout Jira
> > > comments
> > > > or presentations
> > > > 4- A viable alternative exists that appears to have feature parity in
> > > HDFS
> > > > encryption
> > > > 5- HBase has finer grained encryption capabilities that extend beyond
> > > what
> > > > HDFS provides
> > > >
> > > > Moving forward, what's the consensus for supporting this feature?
> > > > Personally, I see two options:
> > > >
> > > > 1- Start going down a path to bring the feature into the forefront
> and
> > > > start providing feature parity with HBase
> > > >
> > > > or
> > > >
> > > > 2- Remove the feature and place emphasis on upstream encryption
> offerings
> > > >
> > > > Any input is welcomed & appreciated!
> > > >
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message