incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Di Spaltro <dan.dispal...@gmail.com>
Subject Re: Binary Loading Question
Date Sun, 04 Oct 2009 22:38:26 GMT
Chris,

Sorry for the delay.  Jbellis figured it out, I am using the OPP and
during serialization the Token was null and throwing.

I will say after looking through the contrib code, the BMT is pretty
odd and especially the way it gets serialized with indexes; then the
payload dropped into the value and the column name is loaded with the
actual ColumnFamily name; however, we have about half a billion points
to migrate and ended up getting like 35k sec from one pretty weak
machine.

Thanks for the help,

On Wed, Sep 30, 2009 at 7:15 PM, Chris Goffinet <cg@chrisgoffinet.com> wrote:
> It would be helpful if you could attach your example. Also did you look at
> the example in contrib by chance?
>
>
> On Sep 30, 2009, at 2:05 PM, Dan Di Spaltro wrote:
>
>> Okay i get what its doing now... its pretty confusing until I saw
>> jbellis' comments.  now when i flush binary it is throwing on the
>> key.toString line because the token is null.  When i setup the
>> rowmutation on the client i used StringToken as opposed to big integer
>> but thats because i am not sure what to put there so i used the tokens
>> chose by cassandra
>>
>> On 9/30/09, Chris Goffinet <cg@chrisgoffinet.com> wrote:
>>>
>>> It's very jank. I kinda understand where they were coming from on that.
>>> Let
>>> me think through it some more and open a  ticket to improve some of that
>>> code.
>>>
>>>
>>> On Wed, Sep 30, 2009 at 12:22 PM, Jonathan Ellis <jbellis@gmail.com>
>>> wrote:
>>>
>>>> So it looks like it takes multiple rows, one per "column" in the
>>>> rowmutation.
>>>>
>>>> Which is confusing and weird if you ask me.
>>>>
>>>> On Wed, Sep 30, 2009 at 1:57 PM, Jonathan Ellis <jbellis@gmail.com>
>>>> wrote:
>>>>>
>>>>> Yeah, just eyeballing it
>>>>>
>>>>>              ColumnFamilyStore cfStore =
>>>>> columnFamilyStores_.get(columnFamily.name());
>>>>>
>>>>> would make more sense to me.
>>>>>
>>>>> Got any insight for us, Chris? :)
>>>>>
>>>>> -Jonathan
>>>>>
>>>>> On Wed, Sep 30, 2009 at 1:47 PM, Dan Di Spaltro
>>>>> <dan.dispaltro@gmail.com>
>>>>
>>>> wrote:
>>>>>>
>>>>>> I was trying to binary load some data into an instance and ran into
a
>>>>>> null exception in the process.  So I began looking at the code,
and it
>>>>>> looks like around Table.java:653 of trunk, that piece of code doesn't
>>>>>> really make sense.  I don't understand the internals of Cassandra
all
>>>>>> that well, but this doesn't seem right.  The null exception came
from
>>>>>> null cfStore.
>>>>>>
>>>>>> Either way I am just trying to use the binarymemtable option to load
>>>>>> values into cassandra, since I have about 240 million rows to load.
>>>>>>
>>>>>> {
>>>>>>              ColumnFamilyStore cfStore =
>>>>>> columnFamilyStores_.get(new String(column.name(), "UTF-8"));
>>>>>>              cfStore.applyBinary(key, column.value());
>>>>>> }
>>>>>>
>>>>>> --
>>>>>> Dan Di Spaltro
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Dan Di Spaltro
>
>



-- 
Dan Di Spaltro

Mime
View raw message