Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of jbellis@gmail.com designates
 74.125.82.172 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=opdqGDkIBdaEbt7Y+hzhVGaMSXZguSFV/Q+qX/V7roQ2lGJ9oc7+JLjqyAKQXKcG0w
         dJDO2SnXkUwNYJvjmRoc/D2oAv7frPVXs7wbeYitXzDk0OVXtl3sUMyW/9DJcAJ9L2aI
         xRxOLoCoT4dTwKaRP3GJyBT/Gr2HIpYHgUpe4=
MIME-Version: 1.0
In-Reply-To: <s2n72c7a7d41005031815g83880f2egae223c9341bd29fa@mail.gmail.com>
References: <l2o72c7a7d41005030349h74b24c63wbfbf6a1f3aea26cd@mail.gmail.com>
	<h2je06563881005030840k214312d6w7b3b6567e958348e@mail.gmail.com>
	<s2n72c7a7d41005031815g83880f2egae223c9341bd29fa@mail.gmail.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Mon, 3 May 2010 23:27:08 -0500
Message-ID: <x2ie06563881005032127vfb9a0da0jeecab47be144293f@mail.gmail.com>
Subject: Re: Feeding in specific Cassandra columns into Hadoop
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

We serialize the SlicePredicate as part of the Hadoop Configuration
string.  It's quite possible that either

 - one of your column names is exposing a bug in the Thrift json serializer
 - Hadoop is silently truncating large predicates

You should test that getSlicePredicate(conf).equals(originalPredicate)

On Mon, May 3, 2010 at 8:15 PM, Mark Schnitzius
<mark.schnitzius@cxense.com> wrote:
> If I take the exact same SlicePredicate that fails in the Hadoop example,
> and pass it in to a=A0multiget_slice, the data is returned successfully. =
=A0So
> it appears the problem does lie somewhere in the tie-in to Hadoop.
> I will try to create a maximally-trimmed-down example that's complete eno=
ugh
> to run on its own that demonstrates the failure, and will post here. =A0I=
 was
> just hoping that there might've been an easy fix recognizable from my
> description before I had to resort to that...
>
> Thanks
> Mark
>
>
> On Tue, May 4, 2010 at 1:40 AM, Jonathan Ellis <jbellis@gmail.com> wrote:
>>
>> Can you reproduce outside the Hadoop environment, i.e. w/ Thrift code?
>>
>> On Mon, May 3, 2010 at 5:49 AM, Mark Schnitzius
>> <mark.schnitzius@cxense.com> wrote:
>> > Hi all... =A0I am trying to feed a specific list of Cassandra column n=
ames
>> > in
>> > as input to a Hadoop process, but for some reason it only feeds in som=
e
>> > of
>> > the columns I specify, not all.
>> > This is a short description of the problem - I'll see if anyone might
>> > have
>> > some insight before I dump a big load of code on you...
>> > 1. =A0I've uploaded a bunch of data into Cassandra; the column names a=
s
>> > longs
>> > (dates, basically) converted to byte[8].
>> > 2. =A0I can successfully set a SlicePredicate using setSlice_range to
>> > return
>> > all the data for a set of columns.
>> > 3. =A0However, if I instead call setColumn_names on the SlicePredicate=
,
>> > only
>> > some of the specified columns get fed into Hadoop.
>> > 4. =A0This faulty behavior is repeatable, with the same columns going
>> > missing
>> > each time for the same input parameters.
>> > 5. =A0For the values that fail, I've made fairly certain that the valu=
e
>> > for
>> > the column name is getting inserted successfully, and that the exact
>> > same
>> > column name is specified in the call to setColumn_names.
>> > Any clues?
>> >
>> > AdTHANKSvance,
>> > Mark
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>
>


--=20
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com