accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Tubbs (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-3163) Mutation could (maybe) benefit from additional validity checks
Date Tue, 23 Sep 2014 18:40:34 GMT


Christopher Tubbs commented on ACCUMULO-3163:

This is a "maybe", because random data corruption can't really be protected against perfectly
(it could always get corrupt after we finish deserializing...), and the RPC layer should already
have some checksums on the data (TCP checksum).

> Mutation could (maybe) benefit from additional validity checks
> --------------------------------------------------------------
>                 Key: ACCUMULO-3163
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>         Environment: Accumulo 1.4
>            Reporter: Christopher Tubbs
> In a 1.4 instance, a mutation was apparently corrupted when it was received by the tserver.
It then got written to the WAL in a corrupted form, and the tserver (likely, as far as can
be discerned) failed to deserialize the data to insert it into the in-memory tree. At some
point this tablet went offline, and failed to be assigned, because WAL recovery could not
complete (specifically, an ArrayIndexOutOfBoundsException, because the length of a column
qualifier was much larger than it should have been, and much larger than the array).
> To protect against corrupt data entering the WAL, the client could provide a fast checksum
on the array, to validate its contents before writing to the WAL.
> The lengths of the fields could be checked for sanity when deserializing, and throw a
more informative error than index out of bounds.

This message was sent by Atlassian JIRA

View raw message