cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From horschi <hors...@gmail.com>
Subject Re: Stale value appears after consecutive TRUNCATE
Date Mon, 08 Aug 2016 09:36:37 GMT
Hi Yuji,

can you reproduce the behaviour with a single node?

The reason I ask is because I probably have the same issue with my
automated tests (which run truncate between every test), which run on my
local laptop.

Maybe around 5 tests randomly fail out of my 1800. I can see that the
failed tests sometimes show data from other tests, which I think must be
because of a failed truncate. This behaviour is seems to be CQL only, or at
least has gotten worse with CQL. I did not experience this with Thrift.

regards,
Christian



On Mon, Aug 8, 2016 at 7:34 AM, Yuji Ito <yuji@imagine-orb.com> wrote:

> Hi all,
>
> I have a question about clearing table and commit log replay.
> After some tables were truncated consecutively, I got some stale values.
> This problem doesn't occur when I clear keyspaces with DROP (and CREATE).
>
> I'm testing the following test with node failure.
> Some stale values appear at checking phase.
>
> Test iteration:
> 1. initialize tables as below
> 2. request a lot of read/write concurrently
> 3. check all records
> 4. repeat from the beginning
>
> I use C* 2.2.6. There are 3 nodes (replication_factor: 3).
> Each node kills cassandra process at random intervals and restarts it
> immediately.
>
> My initialization:
> 1. clear tables with TRUNCATE
> 2. INSERT initial records
> 3. check if all values are correct
>
> If any phase fails (because of node failure), the initialization starts
> all over again.
> So, tables are sometimes truncated consecutively.
> Though the check in the initialization is OK, stale data appears when I
> execute "SELECT * FROM mykeyspace.mytable;" after a lot of requests are
> completed.
>
> The problem is likely to occur when the ReplayPosition's value in
> "truncated_at" is initialized as below after an empty table is truncated.
>
> Column Family ID: truncated_at
> XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX: 0xffffffffffffffff000000000000
> 0156597cd4c7
> (this value was acquired just after phase 1 in my initialization)
>
> I guess some unexpected replays occur.
> Does anyone know the behavior?
>
> Thanks,
> Yuji
>

Mime
View raw message