hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-10885) Support visibility expressions on Deletes
Date Tue, 01 Apr 2014 10:26:15 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13956348#comment-13956348
] 

Andrew Purtell edited comment on HBASE-10885 at 4/1/14 10:25 AM:
-----------------------------------------------------------------

bq. Delete.setCellVisibility() should be supported now.

Yes.

bq. And these labels passed here will be only a list of labels and not visibility expressions
like A|B!C?

No. Deletes should support visibility expressions just like Put, etc. The supplied visibility
expression is then associated with the delete marker(s).

Actually, scratch what I said above in the first comment. I think we can check that the supplied
expression does not exceed the maximal authorization set for the user submitting the Delete
in the preDelete hook and then store the delete marker as a cell with a visibility expression
tag and do the rest of the work later, by hooking the compaction scanner. The big change then
would be checking for visibility expressions in tags on delete markers at compaction time.
If we find one, then we have to filter only the cells covered by the tombstone that have a
matching expression.

If we are not storing visibility expression terminals (LeafExpressionNodes) in sorted order
by ordinal we probably should consider it. (I don't think we are.) Because e.g. A|B == B|A.
It would be most efficient if we can simply do byte comparison of serialized visibility expressions
on the delete marker and any found while enumerating cells covered by it. 

If a delete marker has a visibility expression, then we only apply it to cells with matching
visibility  expressions. If a cell has no visibility tag then it does not match. (A|B != nil)


was (Author: apurtell):
bq. Delete.setCellVisibility() should be supported now.

Yes.

bq. And these labels passed here will be only a list of labels and not visibility expressions
like A|B!C?

No. Deletes should support visibility expressions just like Put, etc. The supplied visibility
expression is then associated with the delete marker(s).

Actually, scratch what I said above in the first comment. I think we can check that the supplied
expression does not exceed the maximal authorization set for the user submitting the Delete
in the preDelete hook and then store the delete marker as a cell with a visibility expression
tag and do the rest of the work later, by hooking the compaction scanner. The big change then
would be checking for visibility expressions in tags on delete markers at compaction time.
If we find one, then we have to filter only the cells covered by the tombstone that have a
matching expression.

If we are not storing visibility expression terminals (LeafExpressionNodes) in sorted order
by ordinal we probably should consider it. (I don't think we are.) Because e.g. A|B == B|A.
It would be most efficient if we can simply do byte comparison of serialized visibility expressions.


> Support visibility expressions on Deletes
> -----------------------------------------
>
>                 Key: HBASE-10885
>                 URL: https://issues.apache.org/jira/browse/HBASE-10885
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.98.1
>            Reporter: Andrew Purtell
>             Fix For: 0.99.0, 0.98.2
>
>
> Accumulo can specify visibility expressions for delete markers. During compaction the
cells covered by the tombstone are determined in part by matching the visibility expression.
This is useful for the use case of data set coalescing, where entries from multiple data sets
carrying different labels are combined into one common large table. Later, a subset of entries
can be conveniently removed using visibility expressions.
> Currently doing the same in HBase would only be possible with a custom coprocessor. Otherwise,
a Delete will affect all cells covered by the tombstone regardless if they are visible to
the user issuing the delete or not. This is correct behavior in that no data spill is possible,
but certainly could be surprising, and is only meant to be transitional. We decided not to
support visibility expressions on Deletes to control the complexity of the initial implementation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message