incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: get_range_slice() tester
Date Mon, 08 Feb 2010 22:58:35 GMT
I'm seeing failures on 0.5 but success against trunk, is that also what you see?

-Jonathan

On Mon, Feb 8, 2010 at 4:42 PM, Jack Culpepper <jackculpepper@gmail.com> wrote:
> On Mon, Feb 8, 2010 at 2:34 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>> This is supposed to pass on a single node but fail on two, correct?
>
> Yep! At least, it does for me.
>
>> What are the tokens on your two nodes, in case that is relevant?
>> (nodeprobe ring will tell you.)
>
> Heh, unfortunately this also shows the fact that I accidentally
> blasted one of my data dirs. ;)
>
> $ sudo bin/nodeprobe -host localhost ring
> [sudo] password for jack:
> Address       Status     Load          Range
>           Ring
>                                       YQVhw0uDS4RMOASI
> 10.212.87.165 Up         8.18 KB       13DyIzn2EhRAHOq9
>           |<--|
> 10.212.230.176Up         11.71 GB      YQVhw0uDS4RMOASI
>           |-->|
>
> J
>
>> -Jonathan
>>
>> On Mon, Feb 8, 2010 at 4:01 PM, Jack Culpepper <jackculpepper@gmail.com> wrote:
>>> Here's a tester program, for contrib. It generates 10 keys using uuid,
>>> inserts them both into the cassandra column family Keyspace1/Super1
>>> and a python dictionary. Then, it does a range scan using both methods
>>> and marks the keys that are returned. Finally, it goes through the
>>> python dictionary, makes sure a cassandra get() on each key works
>>> (should through an exception on failure), and complains about keys
>>> that were not found in the range scan.
>>>
>>> To run, put the contents in test_bug.py then run like this:
>>>
>>> python test_bug.py get_key_range
>>>
>>> (Nothing printed means it worked.)
>>>
>>> python test_bug.py get_range_slice
>>>
>>> (Keys that should have been found in a range scan, but were not, are printed.)
>>>
>>> Best,
>>>
>>> Jack
>>>
>>>
>>>
>>> import sys
>>> import time
>>> import uuid
>>>
>>> from thrift import Thrift
>>> from thrift.transport import TTransport
>>> from thrift.transport import TSocket
>>> from thrift.protocol.TBinaryProtocol import TBinaryProtocolAccelerated
>>> from cassandra import Cassandra
>>> from cassandra.ttypes import *
>>>
>>> num_keys = 10
>>>
>>> socket = TSocket.TSocket("10.212.87.165", 9160)
>>> transport = TTransport.TBufferedTransport(socket)
>>> protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport)
>>> client = Cassandra.Client(protocol)
>>>
>>> ks = "Keyspace1"
>>> cf = "Super1"
>>> cl = ConsistencyLevel.ONE
>>>
>>> d = {}
>>>
>>> transport.open()
>>>
>>> if 1:
>>>    ## insert keys using the raw thrift interface
>>>    cpath = ColumnPath(cf, "foo", "is")
>>>    value = "cool"
>>>
>>>    for i in xrange(num_keys):
>>>        ts = time.time()
>>>        key = uuid.uuid4().hex
>>>        client.insert(ks, key, cpath, value, ts, cl)
>>>        d[key] = 1
>>>
>>> else:
>>>    ## insert keys using pycassa!
>>>    import pycassa
>>>
>>>    client = pycassa.connect(["10.212.87.165:9160"])
>>>    cf_test = pycassa.ColumnFamily(client, ks, cf, super=True)
>>>
>>>    for i in xrange(num_keys):
>>>        key = uuid.uuid4().hex
>>>        cf_test.insert(key, { 'params' : { 'is' : 'cool' }})
>>>        d[key] = 1
>>>
>>>
>>> cparent = ColumnParent(column_family=cf)
>>> slice_range = SliceRange(start="key", finish="key")
>>> p = SlicePredicate(slice_range=slice_range)
>>>
>>> done = False
>>> seg = 1000
>>> start = ""
>>>
>>> ## do a scan using either get_key_range() (deprecated) or get_range_slice()
>>> ## for every key returned that is in the dictionary, mark it as found
>>> while not done:
>>>    if sys.argv[1] == "get_key_range":
>>>        result = client.get_key_range(ks, cf, start, "", seg, cl)
>>>
>>>        if len(result) < seg: done = True
>>>        else: start = result[seg-1]
>>>
>>>        for r in result:
>>>            if d.has_key(r):
>>>                d[r] = 0
>>>
>>>    if sys.argv[1] == "get_range_slice":
>>>        result = client.get_range_slice(ks, cparent, p, start, "", seg, cl)
>>>
>>>        if len(result) < seg: done = True
>>>        else: start = result[seg-1].key
>>>
>>>        for r in result:
>>>            if d.has_key(r.key):
>>>                d[r.key] = 0
>>>
>>> cpath = ColumnPath(column_family=cf, super_column='foo')
>>>
>>> ## get, remove all the keys
>>> ## print all the keys that were not marked 0
>>> for k in d:
>>>    result = client.get(ks, k, cpath, cl)
>>>    #print result
>>>
>>>    if d[k] == 1:
>>>        print k, "not marked 0"
>>>    #else:
>>>    #    print k, "was marked 0!"
>>>
>>>    ts = time.time()
>>>    client.remove(ks, k, cpath, ts, cl)
>>>
>>
>

Mime
View raw message