accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-2925) Timestamp is not propagated to peer
Date Fri, 20 Jun 2014 01:26:25 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038225#comment-14038225
] 

ASF subversion and git services commented on ACCUMULO-2925:
-----------------------------------------------------------

Commit b062a0bd3ed388f89bc04dfa2903bf3cc951976c in accumulo's branch refs/heads/master from
[~elserj]
[ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=b062a0b ]

ACCUMULO-2925 Create regular Mutations from ServerMutations when applying replication data
on a peer

Mutations do not store unserialized ColumnUpdates, but only generate them
on demand via the getter. This is intended to create an efficient implementation
(both performance and size) while preseving immutability.

Server-assigned timestamps work around this immutability by wrapping normal
Mutations in a ServerMutation and ColumnUpdates with ServerColumnUpdates. By doing
this, ServerMutations can "fake" the timestamp on ColumnUpdates that otherwise
do not have a timestamp set.

In the context of replication, this is still a problem as all Mutations that are
sent to a peer are ServerMutations (as we read them from a WAL). These Mutations are
deserialized and passed into a BatchWriter to apply to the local instance; however, the
BatchWriter is ignorant of ServerMutations and the special timestamp handling.

When the BatchWriter makes a "copy" of the Mutation (see ACCUMULO-2915), despite this
being a shallow copy, the server-assigned timestamp is lost by creating a regular
Mutation from what was a ServerMutation. Even if this were possible, the TMutation
class, which the BatchWriter eventually uses to send to the Mutations to a TabletServer,
is also ignorant of the ServerMutation timestamp without modification of the serialization
and TMutation class.

As such, the only option left is to, when encountering ServerMutations in the BatchWriterReplicationReplayer
code, we *must* recreate new Mutations, applying the possibly present server-timestamp to
each new Mutation we create to ensure that the timestamp is correctly propagated to this peer.


> Timestamp is not propagated to peer
> -----------------------------------
>
>                 Key: ACCUMULO-2925
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2925
>             Project: Accumulo
>          Issue Type: Bug
>          Components: replication
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Blocker
>             Fix For: 1.7.0
>
>
> Wrote a test that was doing some more intense verification of equality of two tables
and I was surprised to find that the tables were in fact not equal.
> Digging into it some more, I eventually found that the keys and values were identical,
save for the timestamp. Despite the Mutations coming from the local WAL having timestamps
set by the server, these got lost.
> Specifically, the "real" timestamp is stored on the ServerMutation, not each ColumnUpdate.
On the peer, when the BatchWriter makes a shallow copy of the (Server)Mutation to apply on
the target table for replication, we lose that ServerMutation and get a "regular" Mutation
which has updates that don't have any timestamp set. If the BatchWriter didn't make the shallow
copy, this should work.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message