hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6222) Add per-KeyValue Security
Date Wed, 07 Nov 2012 19:16:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492608#comment-13492608

Andrew Purtell commented on HBASE-6222:

Thanks for attaching the design documents. I could comment on them but am pursing a substantially
different direction.

Regarding KV tags:

Right now I'm experimenting with code that adds tags to KVs in a backwards compatible way.
I would call this a transitional change. We overload the existing KV binary format, technically
the tags are prepended to value data but are not considered part of it. For KVs that contain
tags, the value length is encoded as a negative integer. When a negative value length is encountered,
we walk over the value part of the KV, parsing out tags, until we reach an end marker. Once
we know the size of the tag data, we adjust the value offset and value length returned to
API callers. The parsing of tags in the KV is cached after the first time it is performed.
Thus no change to KV handling has to happen outside of the KeyValue class.

Regarding cell level permissions:

Instead of visibility labels we extend the AccessController to support per cell ACLs. These
are serialized TablePermissions stored with a KV in a tag. We add a helper class UserTablePermissions
which supports both PB and Writable serialization of Listmultimap<String,TablePermission>.
The permissions checking of the AccessController is extended in a straightforward way to check
if any KV passing through the AccessControlFilter contains tags containing ACLs. If so, an
access grant check is first performed using TablePermissions deserialized from the tag(s).
This allows per cell ACLs to overload CF or table level permissions wherever they might exist.
While the TablePermission type is not strictly needed for this (Permission would suffice),
it was easier to simply allow a TablePermission to store nulls for table, cf, and qualifier
than make more extensive changes to the AccessController. Doing so could be considered a refinement.
This does mean that the AccessControlFilter implementation is changed to pass back SKIP instead
of NEXT_COL hints, so every KV is examined by the filter, and use of the filter to wrap Gets
and Scans is now unconditional. The performance impact of this has yet to be quantified but
is expected not to be that different from changes that require per-KV visibility labels to
be evaluated (and could be less expensive since we are not evaluating a mini label language).

No API changes are necessarily required. The new UserTablePermissions type is serialized to
PB and then passed as an attribute using the existing API facility for that. Not all mutation
types are supported. It doesn't make sense to attach an ACL on a Delete, because the KVs covered
by the delete will be ... deleted. Additionally, Increment is not currently supported because
it doesn't support attributes (doesn't extend either Mutation nor OperationWithAttributes)
and anyway the upsert code paths don't pass KVs through a coprocessor down into HBase core
so bundling tags with the user value doesn't work there. Also, while I haven't done this quite
yet, I anticipate the upsert code will require a second core modification to preserve any
tags that might be in the value to update.

Obviously my goal is to add per-cell ("per-KeyValue") security through as simple and straightforward
extensions of the existing AccessController implementation as possible, with minimal to no
changes to core code*. I do not aim for Accumulo parity nor support for multi label security
schemes. It may be more appropriate to post this work to another JIRA when it is ready for
review. This work could be considered orthogonal to any label based approach, and could probably
play nicely together with it should one come to exist, as long as both are based on a generic
KV tag facility.

* - I am also evaluating an option that avoids making any change to KeyValue, thus no change
to core HBase at all. Instead it stores the equivalent of KV tags in "shadow KVs" stored in
a shadow metacolumn. Implementation of this is underway but as you can imagine it's not so
straightforward to produce something that could perform (nearly) as well as inline in-KV checks.
This approach wouldn't have the limitation regarding Increment nor the requirement to change
the upsert code.

All of the above is subject to change.
> Add per-KeyValue Security
> -------------------------
>                 Key: HBASE-6222
>                 URL: https://issues.apache.org/jira/browse/HBASE-6222
>             Project: HBase
>          Issue Type: New Feature
>          Components: security
>            Reporter: stack
>            Assignee: Andrew Purtell
>         Attachments: HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx
> Saw an interesting article: http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14
> "The  Senate Armed Services Committee version of the fiscal 2013 national defense authorization
act (S. 3254) would require DoD agencies to foreswear the Accumulo NoSQL database after Sept.
30, 2013, unless the DoD CIO certifies that there exists either no viable commercial open
source database with security features comparable to [Accumulo] (such as the HBase or Cassandra
> Not sure what a 'commercial open source database' is, and I'm not sure whats going on
in the article, but tra-la-la'ing, if we had per-KeyValue 'security' like Accumulo's, we might
put ourselves in the running for federal contributions?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message