accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-2232) Combiners can cause deleted data to come back
Date Tue, 22 Sep 2015 16:57:04 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902968#comment-14902968
] 

ASF GitHub Bot commented on ACCUMULO-2232:
------------------------------------------

Github user ctubbsii commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/47#discussion_r40111853
  
    --- Diff: core/src/main/java/org/apache/accumulo/core/iterators/Combiner.java ---
    @@ -149,6 +160,38 @@ public void next() throws IOException {
     
       private Key workKey = new Key();
     
    +  private static final Cache<String,Long> loggedMsgCache = CacheBuilder.newBuilder().expireAfterWrite(1,
TimeUnit.HOURS).maximumSize(10000).build();
    +
    +  private void sawDelete() {
    +    if (isPartialCompaction) {
    +      switch (deleteHandlingAction) {
    +        case LOG_ERROR:
    +          try {
    +            loggedMsgCache.get(this.getClass().getName(), new Callable<Long>()
{
    --- End diff --
    
    At first, I thought "Callable<Void>" might be better, but apparently CacheBuilder
doesn't like that, so might be better to use "Callable<Boolean>", because it's more
readable to see "Boolean.TRUE" inserted into the cache to denote that a log message has recently
occurred. (Unless you were going to use the integer to denote how many times it has been logged
before suppressing repeats after some threshold, like 5.)


> Combiners can cause deleted data to come back
> ---------------------------------------------
>
>                 Key: ACCUMULO-2232
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2232
>             Project: Accumulo
>          Issue Type: Bug
>          Components: client, tserver
>            Reporter: John Vines
>
> The case-
> 3 files with-
> * 1 with a key, k, with timestamp 0, value 3
> * 1 with a delete of k with timestamp 1
> * 1 with k with timestamp 2, value 2
> The column of k has a summing combiner set on it. The issue here is that depending on
how the major compactions play out, differing values with result. If all 3 files compact,
the correct value of 2 will result. However, if 1 & 3 compact first, they will aggregate
to 5. And then the delete will fall after the combined value, resulting in the result 5 to
persist.
> First and foremost, this should be documented. I think to remedy this, combiners should
only be used on full MajC, not not full ones. This may necessitate a special flag or a new
combiner that implemented the proper semantics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message