Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=content-type
	:mime-version:subject:from:in-reply-to:date
	:content-transfer-encoding:message-id:references:to; q=dns; s=
	thelastpickle.com; b=hidcBDmb/7S0TJ9QyIeLmwOtA1gE8IB46LAMbCu8Hi7
	DGvhKCfH/rRO8PRhQT2u+JtPt/982znASl6bIl4aepWHCycnjNtx6I8oUAg5/eui
	lo0oJqochWchOiPWBFlvQjZsMeXsVrtVYCcsBVodlQCec55FtJH3y48TfVZNxOLU
	=
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?
From: aaron morton <aaron@thelastpickle.com>
In-Reply-To: <BANLkTimRGPyKQKDRjrP0W7OJ9=gXvVU-vA@mail.gmail.com>
Date: Mon, 9 May 2011 21:33:08 +1200
Content-Transfer-Encoding: quoted-printable
Message-Id: <B9808FE5-A82F-418E-AC68-2CDEABE527A1@thelastpickle.com>
References: <BANLkTi=ENiasLZss5vWzSedEbSxbFWr=TQ@mail.gmail.com>
 <ADF7C794-B283-4CF8-85AC-A17BC39E1088@thelastpickle.com>
 <BANLkTim4cRL5ULdoBpX+mzi2e9j-8pay0w@mail.gmail.com>
 <FA1F2EF4-93BE-484A-B1B4-9AD2B7F46820@gmail.com>
 <BANLkTik5ZjKBjguYR2zKnrBOFCqu8=bhDQ@mail.gmail.com>
 <967C4367-1F04-4365-8237-E54DB3C3451F@gmail.com>
 <EBD750C5-3E87-4BA8-A032-C1756FC4E174@thelastpickle.com>
 <BANLkTik7BSvqyjeWLm6C87hkOLsnSLvn=g@mail.gmail.com>
 <5E594C71-8FEF-425B-8BEE-B5DFE7BED9C7@thelastpickle.com>
 <BANLkTim-F7K3pnM=ENNQPVHAx88Xuu34aw@mail.gmail.com>
 <E7A3A0A5-9926-436A-92BC-D56D97AA830A@thelastpickle.com>
 <BANLkTimRGPyKQKDRjrP0W7OJ9=gXvVU-vA@mail.gmail.com>
To: user@cassandra.apache.org

That was my initial thought, just wanted to see if there was anything =
else going on. Sounds like Henrik has a workaround so all is well.=20

Cheers
=20
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 9 May 2011, at 18:10, Jonathan Ellis wrote:

> Strongly suspect that he has invalid unicode characters in his keys.
> 0.6 wasn't as good at validating those as 0.7.
>=20
> On Sun, May 8, 2011 at 8:51 PM, aaron morton <aaron@thelastpickle.com> =
wrote:
>> Out of interest i've done some more digging. Not sure how much more =
I've
>> contributed but here goes...
>> Ran this against an clean v 0.6.12 and it works (I expected it to =
fail on
>> the first read)
>>     client =3D pycassa.connect()
>>     standard1 =3D pycassa.ColumnFamily(client, 'Keyspace1', =
'Standard1')
>>     uni_str =3D u"=E6=95=B0=E6=99=82=E9=96=93"
>>     uni_str =3D uni_str.encode("utf-8")
>>=20
>>     print "Insert row", uni_str
>>     print uni_str, standard1.insert(uni_str, {"bar" : "baz"})
>>     print "Read rows"
>>     print "???", standard1.get("???")
>>     print uni_str, standard1.get(uni_str)
>> Ran that against the current 0.6 head from the command line and it =
works.
>> Run against the code running in intelli J and the code fails as =
expected.
>> Code also fails as expected on 0.7.5
>> At one stage I grabbed the buffer created by fastbinary.encode_binary =
in the
>> python generated batch_mutate_args.write() and it looked like the key =
was
>> correctly utf-8 encoded (matching bytes to the previous utf-8 =
encoding of
>> that string).
>> I've updated the git
>> project https://github.com/amorton/cassandra-unicode-bug
>> Am going to leave it there unless there is interest to keep looking
>> into it.
>> -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>> On 8 May 2011, at 13:31, Jonathan Ellis wrote:
>>=20
>> Right, that's sort of a half-repair: it will repair differences in
>> replies it got, but it won't doublecheck md5s on the rest in the
>> background. So if you're doing CL.ONE reads this is a no-op.
>>=20
>> On Sat, May 7, 2011 at 4:25 PM, aaron morton =
<aaron@thelastpickle.com>
>> wrote:
>>=20
>> I remembered something like that so had a look at
>> RangeSliceResponseResolver.resolve()  in 0.6.12 and it looks like it
>> schedules the repairs...
>>=20
>>            protected Row getReduced()
>>=20
>>            {
>>=20
>>                ColumnFamily resolved =3D
>> ReadResponseResolver.resolveSuperset(versions);
>>=20
>>                ReadResponseResolver.maybeScheduleRepairs(resolved, =
table,
>> key, versions, versionSources);
>>=20
>>                versions.clear();
>>=20
>>                versionSources.clear();
>>=20
>>                return new Row(key, resolved);
>>=20
>>            }
>>=20
>>=20
>> Is that right?
>>=20
>>=20
>> -----------------
>>=20
>> Aaron Morton
>>=20
>> Freelance Cassandra Developer
>>=20
>> @aaronmorton
>>=20
>> http://www.thelastpickle.com
>>=20
>> On 8 May 2011, at 00:48, Jonathan Ellis wrote:
>>=20
>> range_slices respects consistencylevel, but only single-row reads and
>>=20
>> multiget do the *repair* part of RR.
>>=20
>> On Sat, May 7, 2011 at 1:44 AM, aaron morton =
<aaron@thelastpickle.com>
>> wrote:
>>=20
>> get_range_slices() does read repair if enabled (checked
>> DoConsistencyChecksBoolean in the config, it's on by default) so you =
should
>> be getting good reads. If you want belt-and-braces run nodetool =
repair
>> first.
>>=20
>> Hope that helps.
>>=20
>>=20
>> On 7 May 2011, at 11:46, Jeremy Hanna wrote:
>>=20
>> Great!  I just wanted to make sure you were getting the information =
you
>> needed.
>>=20
>> On May 6, 2011, at 6:42 PM, Henrik Schr=C3=B6der wrote:
>>=20
>> Well, I already completed the migration program. Using =
get_range_slices I
>> could migrate a few thousand rows per second, which means that =
migrating all
>> of our data would take a few minutes, and we'll end up with pristine
>> datafiles for the new cluster. Problem solved!
>>=20
>> I'll see if I can create datafiles in 0.6 that are uncleanable in 0.7 =
so
>> that you all can repeat this and hopefully fix it.
>>=20
>>=20
>> /Henrik Schr=C3=B6der
>>=20
>> On Sat, May 7, 2011 at 00:35, Jeremy Hanna =
<jeremy.hanna1234@gmail.com>
>> wrote:
>>=20
>> If you're able, go into the #cassandra channel on freenode (IRC) and =
talk to
>> driftx or jbellis or aaron_morton about your problem.  It could be =
that you
>> don't have to do all of this based on a conversation there.
>>=20
>> On May 6, 2011, at 5:04 AM, Henrik Schr=C3=B6der wrote:
>>=20
>> I'll see if I can make some example broken files this weekend.
>>=20
>>=20
>> /Henrik Schr=C3=B6der
>>=20
>> On Fri, May 6, 2011 at 02:10, aaron morton <aaron@thelastpickle.com> =
wrote:
>>=20
>> The difficulty is the different thrift clients between 0.6 and 0.7.
>>=20
>> If you want to roll your own solution I would consider:
>>=20
>> - write an app to talk to 0.6 and pull out the data using keys from =
the
>> other system (so you know can check referential integrity while you =
are at
>> it). Dump the data to flat file.
>>=20
>> - write an app to talk to 0.7 to load the data back in.
>>=20
>> I've not given up digging on your migration problem, having to =
manually dump
>> and reload if you've done nothing wrong is not the best solution. =
I'll try
>> to find some time this weekend to test with:
>>=20
>> - 0.6 server, random paritioner, standard CF's, byte column
>>=20
>> - load with python or the cli on osx or ubuntu (dont have a window =
machine
>> any more)
>>=20
>> - migrate and see whats going on.
>>=20
>> If you can spare some sample data to load please send it over in the =
user
>> group or my email address.
>>=20
>> Cheers
>>=20
>> -----------------
>>=20
>> Aaron Morton
>>=20
>> Freelance Cassandra Developer
>>=20
>> @aaronmorton
>>=20
>> http://www.thelastpickle.com
>>=20
>> On 6 May 2011, at 05:52, Henrik Schr=C3=B6der wrote:
>>=20
>> We can't do a straight upgrade from 0.6.13 to 0.7.5 because we have =
rows
>> stored that have unicode keys, and Cassandra 0.7.5 thinks those rows =
in the
>> sstables are corrupt, and it seems impossible to clean it up without =
losing
>> data.
>>=20
>> However, we can still read all rows perfectly via thrift so we are =
now
>> looking at building a simple tool that will copy all rows from our =
0.6.3
>> cluster to a parallell 0.7.5 cluster. Our question is now how to do =
that and
>> ensure that we actually get all rows migrated? It's a pretty small =
cluster,
>> 3 machines, a single keyspace, a singke columnfamily, ~2 million =
rows, a few
>> GB of data, and a replication factor of 3.
>>=20
>> So what's the best way? Call get_range_slices and move through the =
entire
>> token space? We also have all row keys in a secondary system, would =
it be
>> better to use that and make calls to get_multi or get_multi_slices =
instead?
>> Are we correct in assuming that if we use the consistencylevel ALL =
we'll get
>> all rows?
>>=20
>>=20
>> /Henrik Schr=C3=B6der
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>> --
>>=20
>> Jonathan Ellis
>>=20
>> Project Chair, Apache Cassandra
>>=20
>> co-founder of DataStax, the source for professional Cassandra support
>>=20
>> http://www.datastax.com
>>=20
>>=20
>>=20
>>=20
>>=20
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>>=20
>>=20
>=20
>=20
>=20
> --=20
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com