Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of filippo@ntoklo.com designates
 209.85.215.172 as permitted sender)
Date: Wed, 15 Aug 2012 10:32:09 +0100
From: Filippo Diotalevi <filippo@ntoklo.com>
To: user@cassandra.apache.org
Message-ID: <0E1DF749F2C142B38EA75A47216970A9@ntoklo.com>
Subject: Migrating to a new cluster (using SSTableLoader or other
 approaches)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hi, =20
we are trying to use SSTableLoader to bootstrap a new 7-node cassandra (v=
. 1.0.10) cluster with the snapshots taken from a 3-node cassandra cluste=
r. The new cluster is in a different data centre. =20

After reading the articles at
=5B1=5D http://www.datastax.com/dev/blog/bulk-loading
=5B2=5D http://geekswithblogs.net/johnsPerfBlog/archive/2011/07/26/how-to=
-use-cassandrs-sstableloader.aspx

we are tried to follow this procedure =20
1) we took a snapshot of our keyspaces in the old cluster and moved them =
to the data folder of 3 of the new machines
2) started cassandra in the new cluster
but we noticed that some column families were missing, other had missing =
data.

After that we tried to use sstableloader
1) we reinstalled cassandra in the new cluster
2) run sstableloader (as explained in =5B2=5D) to load the keyspaces

SSTableLoader starts, but the progress is always 0 and the transfer rate =
is 0MB/s. Some warning and exceptions are present in the logs

./sstableloader /opt/analytics/analytics/
Starting client (and waiting 30 seconds for gossip) ...
Streaming revelant part of /opt/analytics/analytics/chart-hd-104-Data.db =
/opt/analytics/analytics/chart-hd-105-Data.db /opt/analytics/analytics/ch=
art-hd-106-Data.db /opt/analytics/analytics/chart-hd-107-Data.db /opt/ana=
lytics/analytics/chart-hd-108-Data.db to =5B/1x.xx.xx.xx5, /1x.xx.xx.xx7,=
 /1x.xx.xx.xx0, /1x.xx.xx.xx7, /1x.xx.xx.xx3, /1x.xx.xx.xx8, /1x.xx.xx.xx=
7=5D
WARN 09:02:38,534 Unable to instantiate cache provider org.apache.cassand=
ra.cache.SerializingCacheProvider; using default org.apache.cassandra.cac=
he.ConcurrentLinkedHashCacheProvider=405d59054d instead
WARN 09:02:38,549 Unable to instantiate cache provider org.apache.cassand=
ra.cache.SerializingCacheProvider; using default org.apache.cassandra.cac=
he.ConcurrentLinkedHashCacheProvider=405d59054d instead


=5B=E2=80=A6.=5D
ERROR 09:02:38,614 Error in ThreadPoolExecutor
java.lang.RuntimeException: java.io.EO=46Exception: unable to seek to pos=
ition 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (65737276=
 bytes) in read-only mode
at org.apache.cassandra.utils.=46BUtilities.unchecked(=46BUtilities.java:=
689)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34=
)
at java.util.concurrent.ThreadPoolExecutor=24Worker.runTask(ThreadPoolExe=
cutor.java:886)
at java.util.concurrent.ThreadPoolExecutor=24Worker.run(ThreadPoolExecuto=
r.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EO=46Exception: unable to seek to position 93069003 in=
 /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in read-o=
nly mode
at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReade=
r.java:253)
at org.apache.cassandra.streaming.=46ileStreamTask.stream(=46ileStreamTas=
k.java:136)
at org.apache.cassandra.streaming.=46ileStreamTask.runMayThrow(=46ileStre=
amTask.java:91)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30=
)
... 3 more
Exception in thread =22Streaming:1=22 java.lang.RuntimeException: java.io=
.EO=46Exception: unable to seek to position 93069003 in /opt/analytics/an=
alytics/chart-hd-104-Data.db (65737276 bytes) in read-only mode
at org.apache.cassandra.utils.=46BUtilities.unchecked(=46BUtilities.java:=
689)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34=
)
at java.util.concurrent.ThreadPoolExecutor=24Worker.runTask(ThreadPoolExe=
cutor.java:886)
at java.util.concurrent.ThreadPoolExecutor=24Worker.run(ThreadPoolExecuto=
r.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EO=46Exception: unable to seek to position 93069003 in=
 /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in read-o=
nly mode
at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReade=
r.java:253)
at org.apache.cassandra.streaming.=46ileStreamTask.stream(=46ileStreamTas=
k.java:136)
at org.apache.cassandra.streaming.=46ileStreamTask.runMayThrow(=46ileStre=
amTask.java:91)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30=
)
... 3 more


---------------


What's the correct approach to migrate data from one cluster to another=3F=
 How can I troubleshoot the problem with sstableloader=3F =20

Thanks,
-- =20
=46ilippo Diotalevi