incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Gallamore <mike.e.gallam...@googlemail.com>
Subject Re: writes to Cassandra failing occasionally
Date Thu, 08 Apr 2010 18:59:21 GMT
On 04/08/2010 05:53 AM, Ted Zlatanov wrote:
> On Thu, 08 Apr 2010 12:53:48 +0100 Philip Jackson<phil@shellarchive.co.uk>  wrote:
>
> PJ>  At Wed, 07 Apr 2010 13:19:26 -0700,
> PJ>  Mike Gallamore wrote:
>    
>>> I have writes to cassandra that are failing, or at least a read shortly
>>> after a write is still getting an old value. I realize Cassandra is
>>> "eventually consistent" but this system is a single CPU single node with
>>> consistency level set to 1, so this seems odd to me.
>>>        
> PJ>  I'm having this problem too (see my post the other day). I use N::C
> PJ>  but generate timestamps in the same way as N::C::E, I've tested that
> PJ>  each is smaller than the next so I'm wondering if I'm barking up the
> PJ>  wrong tree.
>
> PJ>  If you figure out what's going on please do post back here, I'll do
> PJ>  the same.
>
> Please put together a test that runs against the default keyspace
> (Keyspace1) or give me your configuration plus a test.  At the very
> least, set $Net::Cassandra::Easy::DEBUG to 1 and look at the timestamps
> it's generating in the Thrift requests.
>
> By default N::C::Easy uses Moose to provide timestamps through a
> double-wrapped (so it's called every time) sub:
>
> has timestamp =>  ( is =>  'ro', isa =>  'CodeRef', default =>  sub { sub
{ join('', gettimeofday()) } } );
>
> This has worked for me but it certainly could be the problem.
>
> Ted
>
>    
Hello Ted. Thanks for your help. I already looked at that line and 
modified it slightly so that it would output the timestamp  so I could 
inspect it. I didn't notice it, the timestamp seemed to be always 
increasing as the most significant digits were increasing. One of my 
colleagues when looking over my shoulder noticed that the number of 
digits in the timestamp when it failed is different. I think this is a 
bug. A straight join won't work because the microseconds part of the 
Time::HiRes return is of varying length, ie it isn't zero padded so 23s 
25 microseconds would show up as 2325 with the join not 23000025 which 
would mean that it would look older than 22.9999993 for example.

I've written a patch that zero pads the numbers. I've attached it to 
this post but encase attachments don't come through on this mailinglist 
here is the body:

diff -ru Net-Cassandra-Easy-0.09/lib/Net/Cassandra/Easy.pm 
Net-Cassandra-Easy-0.09-patched/lib/Net/Cassandra/Easy.pm
--- Net-Cassandra-Easy-0.09/lib/Net/Cassandra/Easy.pm    2010-04-06 
13:03:59.000000000 -0700
+++ Net-Cassandra-Easy-0.09-patched/lib/Net/Cassandra/Easy.pm    
2010-04-08 11:29:39.000000000 -0700
@@ -40,7 +40,11 @@
  has recv_buffer  => ( is => 'ro', isa => 'Int',     default => 1024 );
  has send_buffer  => ( is => 'ro', isa => 'Int',     default => 1024 );
  has max_results  => ( is => 'ro', isa => 'Int',     default => 100 );
-has timestamp    => ( is => 'ro', isa => 'CodeRef', default => sub { 
sub { join('', gettimeofday()) } } );
+has timestamp    => ( is => 'ro', isa => 'CodeRef', default => sub {
+        sub {
+            my ($secs, $usecs) = gettimeofday();
+            return $secs . sprintf "%0.6d", $usecs;
+        } } );

  # read and write consistency can be changed on the fly
  has read_consistency  => ( is => 'rw', isa => 'Int', default => 
Net::GenCassandra::ConsistencyLevel::ONE );

I'm not sure if it is a setting I need to change or what but I don't see 
my own posts to the user group so if you could reply to this to let me 
know that you saw it it would be great. Otherwise I'll never know if you 
got the patch.

Mime
View raw message