Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 14202705C for ; Wed, 3 Aug 2011 03:00:57 +0000 (UTC) Received: (qmail 56667 invoked by uid 500); 3 Aug 2011 03:00:53 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 56639 invoked by uid 500); 3 Aug 2011 03:00:50 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 56625 invoked by uid 99); 3 Aug 2011 03:00:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Aug 2011 03:00:49 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.210.172] (HELO mail-iy0-f172.google.com) (209.85.210.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Aug 2011 03:00:43 +0000 Received: by iye7 with SMTP id 7so487301iye.31 for ; Tue, 02 Aug 2011 20:00:23 -0700 (PDT) Received: by 10.231.190.204 with SMTP id dj12mr413986ibb.195.1312340422841; Tue, 02 Aug 2011 20:00:22 -0700 (PDT) Received: from [10.0.1.90] (203-114-161-15.wir.sta.inspire.net.nz [203.114.161.15]) by mx.google.com with ESMTPS id v3sm265056ibh.33.2011.08.02.20.00.05 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 02 Aug 2011 20:00:16 -0700 (PDT) Sender: Todd Nine Subject: Re: Input on difficult migration issue from 0.7.4 to 0.8.2 From: Todd Nine To: user@cassandra.apache.org In-Reply-To: References: <1312333592.29615.11.camel@greenlantern.local> Content-Type: multipart/alternative; boundary="=-WqzZqJxeXKa0qhWzvAv9" Date: Wed, 03 Aug 2011 15:00:02 +1200 Message-ID: <1312340402.29615.31.camel@greenlantern.local> Mime-Version: 1.0 X-Mailer: Evolution 2.32.2 --=-WqzZqJxeXKa0qhWzvAv9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Hi Johnathan. Are these the correct steps for the upgrade? 1. Copy SSTable from 0.7.4 snapshot 2. Drop CFS in all nodes in 0.7.4 cluster 3. Upgrade to 0.8.2 and start cluster (with no clients connected) 4. Create CFs using cli in cluster 5. Shutdown all nodes 6. Move the SSTables into place on all the nodes 7. Restart the nodes This would definitely be my preferred approach, however I wasn't sure if SSTables stored the CF meta data. Thanks, Todd On Tue, 2011-08-02 at 21:34 -0500, Jonathan Ellis wrote: > If the bytes are compatible why not just use the existing sstable > files after redefining the schema? > > On Tue, Aug 2, 2011 at 8:06 PM, Todd Nine wrote: > > Hi all, > > I seem to have backed myself into a corner and I cannot easily upgrade > > from a custom 0.7.4 installation to the default 0.8.2. Any help I could get > > would be greatly appreciated. Below is an outline of the problem. > > > > Current installation: > > 0.7.4 with Ed Anuff's custom composite comparators. > > https://github.com/riptano/hector-composite/commit/74407df3ef97c3fa90eae9f68d84b50be3d9938e > > CF Comprator Type : "DynamicComposite" > > > > > > New installation: > > 0.8.2 with Casasndra's native DynamicComposite column time. > > CF Comparator Type: > > "DynamicComposite(a=>AsciiType,b=>BytesType,i=>IntegerType,x=>LexicalUUIDType,l=>LongType,t=>TimeUUIDType,s=>UTF8Type,u=>UUIDType,A=>AsciiType(reversed=true),B=>BytesType(reversed=true),I=>IntegerType(reversed=true),X=>LexicalUUIDType(reversed=true),L=>LongType(reversed=true),T=>TimeUUIDType(reversed=true),S=>UTF8Type(reversed=true),U=>UUIDType(reversed=true))" > > > > > > I cannot simply upgrade Cassandra, this fails because the comparator is > > incorrectly defined for version 0.8.1 and on. My issue is that the column > > family definition has changed, the bytes that are stored are compatible, I > > simply need to re-define the CF and migrate the column data over. > > > > > > Initially I was thinking I could perform a read from the rows and columns > > from a single 0.7 node, then insert them into the 0.8 cluster, however I > > cannot have 2 different versions of the thrift API running in the same java > > JVM due to namespace conflicts. > > > > > > Idea is to perform the following steps, if anyone has any better > > suggestions, they would be greatly appreciated. > > > > 1. Back up all my CFs that use the dynamic composite, and copy the SSTable > > out of my Keyspace's data directory to tmp > > 2. Drop all CFs that use dynamic composite > > 3. Re-create the CFs with the new comparator definition > > 4. Using an external program directly read the 0.7 SSTables (without > > starting the cassandra daemon) and insert the rows and columns into the 0.8 > > cluster via thrift. > > > > > > Can anyone point me at a good example for reading rows and columns directly > > from the SSTables without using the thrift api? > > > > Thanks, > > Todd > > > --=-WqzZqJxeXKa0qhWzvAv9 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 7bit Hi Johnathan.  Are these the correct steps for the upgrade?

1. Copy SSTable from 0.7.4 snapshot
2. Drop CFS in all nodes in 0.7.4 cluster
3. Upgrade to 0.8.2 and start cluster (with no clients connected)
4. Create CFs using cli in cluster
5. Shutdown all nodes
6. Move the SSTables into place on all the nodes
7. Restart the nodes

This would definitely be my preferred approach, however I wasn't sure if SSTables stored the CF meta data.

Thanks,
Todd


On Tue, 2011-08-02 at 21:34 -0500, Jonathan Ellis wrote: 
If the bytes are compatible why not just use the existing sstable
files after redefining the schema?

On Tue, Aug 2, 2011 at 8:06 PM, Todd Nine <todd@spidertracks.com> wrote:
> Hi all,
>   I seem to have backed myself into a corner and I cannot easily upgrade
> from a custom 0.7.4 installation to the default 0.8.2.  Any help I could get
> would be greatly appreciated.  Below is an outline of the problem.
>
> Current installation:
> 0.7.4 with Ed Anuff's custom composite comparators.
> https://github.com/riptano/hector-composite/commit/74407df3ef97c3fa90eae9f68d84b50be3d9938e
> CF Comprator Type : "DynamicComposite"
>
>
> New installation:
> 0.8.2 with Casasndra's native DynamicComposite column time.
> CF Comparator Type:
> "DynamicComposite(a=>AsciiType,b=>BytesType,i=>IntegerType,x=>LexicalUUIDType,l=>LongType,t=>TimeUUIDType,s=>UTF8Type,u=>UUIDType,A=>AsciiType(reversed=true),B=>BytesType(reversed=true),I=>IntegerType(reversed=true),X=>LexicalUUIDType(reversed=true),L=>LongType(reversed=true),T=>TimeUUIDType(reversed=true),S=>UTF8Type(reversed=true),U=>UUIDType(reversed=true))"
>
>
> I cannot simply upgrade Cassandra, this fails because the comparator is
> incorrectly defined for version 0.8.1 and on.  My issue is that the column
> family definition has changed, the bytes that are stored are compatible, I
> simply need to re-define the CF and migrate the column data over.
>
>
> Initially I was thinking I could perform a read from the rows and columns
> from a single 0.7 node, then insert them into the 0.8 cluster, however I
> cannot have 2 different versions of the thrift API running in the same java
> JVM due to namespace conflicts.
>
>
> Idea is to perform the following steps, if anyone has any better
> suggestions, they would be greatly appreciated.
>
> 1. Back up all my CFs that use the dynamic composite, and copy the SSTable
> out of my Keyspace's data directory to tmp
> 2. Drop all CFs that use dynamic composite
> 3. Re-create the CFs with the new comparator definition
> 4. Using an external program directly read the 0.7 SSTables (without
> starting the cassandra daemon) and insert the rows and columns into the 0.8
> cluster via thrift.
>
>
> Can anyone point me at a good example for reading rows and columns directly
> from the SSTables without using the thrift api?
>
> Thanks,
> Todd
--=-WqzZqJxeXKa0qhWzvAv9--