Mailing-List: contact cassandra-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cassandra-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of jbellis@gmail.com designates
 209.85.219.210 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=L87+IJz7Tsw1mzWiiEgRpz5n75sX1ZsCymk+En/JAWCuNfob8KuIMAXr5WQ7k/gAI8
         zBrBdlknaTkPpfN9PjKp017ffd7Hu36BDKyeQVBkkBGB6i19ObCCQWVJn1+ApQbYKP0M
         sT/QLQDczOYKyas5FEr1L0vKkvmDZJilWPmwA=
MIME-Version: 1.0
In-Reply-To: <f05b8c590912071622q79266270h22f3491b6e41938@mail.gmail.com>
References: <f05b8c590912071443x7ec882fcya3d67b3bcc915564@mail.gmail.com>
	<e06563880912071449r4450102fm444657b35affea86@mail.gmail.com>
	<e06563880912071449r42473c06m8ecf783a97e32792@mail.gmail.com>
	<f05b8c590912071456p3156a7cew8895ee9a11972ab2@mail.gmail.com>
	<e06563880912071509l33fb96f9u79e9a316f5e549e2@mail.gmail.com>
	<f05b8c590912071622q79266270h22f3491b6e41938@mail.gmail.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Tue, 8 Dec 2009 21:08:28 -0600
Message-ID: <e06563880912081908i1397c0dahb8d4548cd1538cd3@mail.gmail.com>
Subject: Re: cassandra mangling non-ascii keys
To: cassandra-user@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I don't remember, but it was definitely wrong in hindsight :(

On Mon, Dec 7, 2009 at 6:22 PM, Edmond Lau <edmond@ooyala.com> wrote:
> Ok - so my understanding from reading the two jira issues is that
> python and ruby treat the "string" thrift type as unencoded bytes
> whereas java treats them as utf-8 encoded bytes. =A0What was the
> rationale behind declaring keys to be of type "string" rather than of
> type "binary"? =A0With "binary", presumably java wouldn't treat keys as
> utf-8 encoded bytes.
>
> Edmond
>
> On Mon, Dec 7, 2009 at 3:09 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>> I suspect you will need to explicitly encode to UTF8 first, then.
>> (And decode when reading.)
>>
>> My reading of the relevant issues
>> (https://issues.apache.org/jira/browse/THRIFT-395,
>> https://issues.apache.org/jira/browse/THRIFT-414) is that this won't
>> be fixed any time soon.
>>
>> -Jonathan
>>
>> On Mon, Dec 7, 2009 at 4:56 PM, Edmond Lau <edmond@ooyala.com> wrote:
>>> This particular client was in Ruby.
>>>
>>> On Mon, Dec 7, 2009 at 2:49 PM, Jonathan Ellis <jbellis@gmail.com> wrot=
e:
>>>> (bugs in thrift, that is)
>>>>
>>>> On Mon, Dec 7, 2009 at 4:49 PM, Jonathan Ellis <jbellis@gmail.com> wro=
te:
>>>>> what language are your clients in? =A0there are definitely some bugs
>>>>> there when communicating b/t client and server of different languages=
.
>>>>> :(
>>>>>
>>>>> On Mon, Dec 7, 2009 at 4:43 PM, Edmond Lau <edmond@ooyala.com> wrote:
>>>>>> I'm using non-ascii keys on Cassandra, relatively close to trunk at
>>>>>> r880926, and my some of my keys get mangled.
>>>>>>
>>>>>> As a simple test case, if I insert a one-byte key anywhere between
>>>>>> \200 and \377 (octal for 128 to 255) through the thrift interface, a=
nd
>>>>>> then query back my data with multi get, I get a hash back that has
>>>>>> "\357\277\275" as the key. =A0All those one-byte keys get mapped to =
the
>>>>>> same bucket, so if I insert with the key \205, I get the data back
>>>>>> when querying for \300. =A0So either a) there's a bug in thrift, b)
>>>>>> Cassandra doesn't support non-ascii keys, or c) Cassandra is manglin=
g
>>>>>> my key somewhere.
>>>>>>
>>>>>> Has anyone else run into this issue?
>>>>>>
>>>>>> Edmond
>>>>>>
>>>>>
>>>>
>>>
>>
>