Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Tue, 1 Sep 2015 16:57:46 +0000 (UTC)
From: "Keith Turner (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12690066.1390324899000.224388.1441126666771@Atlassian.JIRA>
In-Reply-To: <JIRA.12690066.1390324899000@Atlassian.JIRA>
References: <JIRA.12690066.1390324899000@Atlassian.JIRA>
 <JIRA.12690066.1390324899643@arcas>
Subject: [jira] [Comment Edited] (ACCUMULO-2232) Combiners can cause deleted
 data to come back
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/ACCUMULO-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725686#comment-14725686 ] 

Keith Turner edited comment on ACCUMULO-2232 at 9/1/15 4:57 PM:
----------------------------------------------------------------

bq. the performance implications will be huge and this is enough rope for people to hang themselves with. However, I think a lot of people use combiners on tables that are append-only and never delete

Thinking about this case where people want to use combiners and do not delete, there is the exception option to consider.
Make combiners throw an exception if a delete marker is seen during a partial major compaction.  However this only makes a user aware of the problem, it does not prevent the problem.  The reason this approach does not prevent the problems is that by the time a delete marker is seen, data that was supposed to have been deleted could have already been combined by a previous partial compaction that did not see any delete markers.


was (Author: kturner):
bq. the performance implications will be huge and this is enough rope for people to hang
themselves with. However, I think a lot of people use combiners on tables
that are append-only and never delete

Thinking about this case where people want to use combiners and do not delete, there is the exception option to consider.
Make combiners throw an exception if a delete marker is seen during a partial major compaction.  However this only makes a user aware of the problem, it does not prevent the problem.  The reason this approach does not prevent the problems is that by the time a delete marker is seen, data that was supposed to have been deleted could have already been combined by a previous partial compaction that did not see any delete markers.

> Combiners can cause deleted data to come back
> ---------------------------------------------
>
>                 Key: ACCUMULO-2232
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2232
>             Project: Accumulo
>          Issue Type: Bug
>          Components: client, tserver
>            Reporter: John Vines
>
> The case-
> 3 files with-
> * 1 with a key, k, with timestamp 0, value 3
> * 1 with a delete of k with timestamp 1
> * 1 with k with timestamp 2, value 2
> The column of k has a summing combiner set on it. The issue here is that depending on how the major compactions play out, differing values with result. If all 3 files compact, the correct value of 2 will result. However, if 1 & 3 compact first, they will aggregate to 5. And then the delete will fall after the combined value, resulting in the result 5 to persist.
> First and foremost, this should be documented. I think to remedy this, combiners should only be used on full MajC, not not full ones. This may necessitate a special flag or a new combiner that implemented the proper semantics.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)