Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of filippo@ntoklo.com designates
 74.125.83.44 as permitted sender)
Date: Thu, 16 Aug 2012 09:51:27 +0100
From: Filippo Diotalevi <filippo@ntoklo.com>
To: user@cassandra.apache.org
Message-ID: <95ECC1ABC58041C8A94503285E016C97@ntoklo.com>
In-Reply-To: <C640984E-E26A-4675-9CA3-CC12E334BEDB@thelastpickle.com>
References: <0E1DF749F2C142B38EA75A47216970A9@ntoklo.com>
 <C640984E-E26A-4675-9CA3-CC12E334BEDB@thelastpickle.com>
Subject: Re: Migrating to a new cluster (using SSTableLoader or other
 approaches)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

> > ERROR 09:02:38,614 Error in ThreadPoolExecutor
> > java.lang.RuntimeException: java.io.EO=46Exception: unable to seek to=
 position 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (6573=
7276 bytes) in read-only mode
> =20
> =20
> This one looks like an error.
> =20
> Can you run nodetool with DEBUG level logging and post the logs =3F =20

Thank Aaron.
 Which nodetool command are you referring to=3F (info, cfstats, ring,=E2=80=
=A6.)
Do I modify the log4j-tools.properties in =24CASSANDRA=5FHOME/conf to set=
 the nodetool logs to DEBUG=3F

Thanks,
-- =20
=46ilippo Diotalevi


On Wednesday, 15 August 2012 at 22:53, aaron morton wrote:

> > WARN 09:02:38,534 Unable to instantiate cache provider org.apache.cas=
sandra.cache.SerializingCacheProvider; using default org.apache.cassandra=
.cache.ConcurrentLinkedHashCacheProvider=405d59054d instead
> =20
> Happens when JNA is not in the path. Nothing to worry about when using =
the sstableloader. =20
> =20
> > ERROR 09:02:38,614 Error in ThreadPoolExecutor
> > java.lang.RuntimeException: java.io.EO=46Exception: unable to seek to=
 position 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (6573=
7276 bytes) in read-only mode
> =20
> This one looks like an error. =20
> =20
> Can you run nodetool with DEBUG level logging and post the logs =3F =20
> =20
> Cheers
> =20
> -----------------
> Aaron Morton
> =46reelance Developer
> =40aaronmorton
> http://www.thelastpickle.com
> =20
> =20
> =20
> =20
> =20
> =20
> On 15/08/2012, at 9:32 PM, =46ilippo Diotalevi <filippo=40ntoklo.com (m=
ailto:filippo=40ntoklo.com)> wrote:
> > Hi, =20
> > we are trying to use SSTableLoader to bootstrap a new 7-node cassandr=
a (v. 1.0.10) cluster with the snapshots taken from a 3-node cassandra cl=
uster. The new cluster is in a different data centre. =20
> > =20
> > After reading the articles at
> > =5B1=5D http://www.datastax.com/dev/blog/bulk-loading
> > =5B2=5D http://geekswithblogs.net/johnsPerfBlog/archive/2011/07/26/ho=
w-to-use-cassandrs-sstableloader.aspx
> > =20
> > we are tried to follow this procedure =20
> > 1) we took a snapshot of our keyspaces in the old cluster and moved t=
hem to the data folder of 3 of the new machines
> > 2) started cassandra in the new cluster
> > but we noticed that some column families were missing, other had miss=
ing data.
> > =20
> > After that we tried to use sstableloader
> > 1) we reinstalled cassandra in the new cluster
> > 2) run sstableloader (as explained in =5B2=5D) to load the keyspaces
> > =20
> > SSTableLoader starts, but the progress is always 0 and the transfer r=
ate is 0MB/s. Some warning and exceptions are present in the logs
> > =20
> > ./sstableloader /opt/analytics/analytics/
> > Starting client (and waiting 30 seconds for gossip) ...
> > Streaming revelant part of /opt/analytics/analytics/chart-hd-104-Data=
.db /opt/analytics/analytics/chart-hd-105-Data.db /opt/analytics/analytic=
s/chart-hd-106-Data.db /opt/analytics/analytics/chart-hd-107-Data.db /opt=
/analytics/analytics/chart-hd-108-Data.db to =5B/1x.xx.xx.xx5, /1x.xx.xx.=
xx7, /1x.xx.xx.xx0, /1x.xx.xx.xx7, /1x.xx.xx.xx3, /1x.xx.xx.xx8, /1x.xx.x=
x.xx7=5D
> > WARN 09:02:38,534 Unable to instantiate cache provider org.apache.cas=
sandra.cache.SerializingCacheProvider; using default org.apache.cassandra=
.cache.ConcurrentLinkedHashCacheProvider=405d59054d instead
> > WARN 09:02:38,549 Unable to instantiate cache provider org.apache.cas=
sandra.cache.SerializingCacheProvider; using default org.apache.cassandra=
.cache.ConcurrentLinkedHashCacheProvider=405d59054d instead
> > =20
> > =20
> > =5B=E2=80=A6.=5D
> > ERROR 09:02:38,614 Error in ThreadPoolExecutor
> > java.lang.RuntimeException: java.io.EO=46Exception: unable to seek to=
 position 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (6573=
7276 bytes) in read-only mode
> > at org.apache.cassandra.utils.=46BUtilities.unchecked(=46BUtilities.j=
ava:689)
> > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.jav=
a:34)
> > at java.util.concurrent.ThreadPoolExecutor=24Worker.runTask(ThreadPoo=
lExecutor.java:886)
> > at java.util.concurrent.ThreadPoolExecutor=24Worker.run(ThreadPoolExe=
cutor.java:908)
> > at java.lang.Thread.run(Thread.java:619)
> > Caused by: java.io.EO=46Exception: unable to seek to position 9306900=
3 in /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in re=
ad-only mode
> > at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessR=
eader.java:253)
> > at org.apache.cassandra.streaming.=46ileStreamTask.stream(=46ileStrea=
mTask.java:136)
> > at org.apache.cassandra.streaming.=46ileStreamTask.runMayThrow(=46ile=
StreamTask.java:91)
> > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.jav=
a:30)
> > ... 3 more
> > Exception in thread =22Streaming:1=22 java.lang.RuntimeException: jav=
a.io.EO=46Exception: unable to seek to position 93069003 in /opt/analytic=
s/analytics/chart-hd-104-Data.db (65737276 bytes) in read-only mode
> > at org.apache.cassandra.utils.=46BUtilities.unchecked(=46BUtilities.j=
ava:689)
> > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.jav=
a:34)
> > at java.util.concurrent.ThreadPoolExecutor=24Worker.runTask(ThreadPoo=
lExecutor.java:886)
> > at java.util.concurrent.ThreadPoolExecutor=24Worker.run(ThreadPoolExe=
cutor.java:908)
> > at java.lang.Thread.run(Thread.java:619)
> > Caused by: java.io.EO=46Exception: unable to seek to position 9306900=
3 in /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in re=
ad-only mode
> > at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessR=
eader.java:253)
> > at org.apache.cassandra.streaming.=46ileStreamTask.stream(=46ileStrea=
mTask.java:136)
> > at org.apache.cassandra.streaming.=46ileStreamTask.runMayThrow(=46ile=
StreamTask.java:91)
> > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.jav=
a:30)
> > ... 3 more
> > =20
> > =20
> > =20
> > =20
> > =20
> > ---------------
> > =20
> > =20
> > What's the correct approach to migrate data from one cluster to anoth=
er=3F How can I troubleshoot the problem with sstableloader=3F =20
> > =20
> > Thanks,
> > -- =20
> > =46ilippo Diotalevi
> =20