incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yan Chunlu <springri...@gmail.com>
Subject Re: progress of sstableloader keeps 0?
Date Mon, 26 Sep 2011 03:30:10 GMT
thank you very much aaron. your explanation  is clear enough and very
helpful!

On Mon, Sep 26, 2011 at 4:58 AM, aaron morton <aaron@thelastpickle.com>wrote:

> If you had RF3 in a 3 node cluster and everything was repaired you *should*
> be ok to only take the data from 1 node, if the cluster is not receiving
> writes.
>
> If you want to merge the data from 3 nodes rename the files AFAIK they do
> not have to have contiguous file numbers.
>
> Cheers
>
>
>  -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 25/09/2011, at 10:45 PM, Yan Chunlu wrote:
>
> thanks!  another problem is what if cluster number are not the same?
>
> in my case I am move 3 nodes cluster data to 1 node,  the keyspace files in
> 3 nodes might use the same name...
>
> I am using the new cluster only for emergency usage, so only 1 node is
> attached.
>
> On Sun, Sep 25, 2011 at 5:20 PM, aaron morton <aaron@thelastpickle.com>wrote:
>
>> That can read data from previous versions, i.e. if you upgrade to 0.8 it
>> can read the existing files from 0.7.
>>
>> But what you are doing with the sstable loader is (AFAIK) only copying the
>> Data portion of the CF. Once the table is loaded the node will then build
>> the Index and the Filter, this is the createBuild() call in the stack. It's
>> throwing because version 0.8 does not want to make version 0.8 Index and and
>> Filter files for a version 0.7 Data file.
>>
>> We get the same problem when upgrading from 0.7 to 0.8, where Repair will
>> not work because it is streaming a 0.7 version data file and the recipient
>> then tries to build the Index and Filter files.
>>
>> So to read 0.7 data from 0.8 just copy over *all* the files for the
>> keyspace (data, filter and index). Then scrub the nodes so that repair can
>> work.
>>
>> Hope that helps.
>>
>>
>>  -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 25/09/2011, at 6:07 PM, Yan Chunlu wrote:
>>
>> yes, I did.  thought 0.8 is downward compatible. is there other ways to
>> load 0.7's data into 0.8?  will copy the data dir directly will work?   I
>> would like to put load of three nodes into one node.
>>
>>  thanks!
>>
>> On Sun, Sep 25, 2011 at 11:52 AM, aaron morton <aaron@thelastpickle.com>wrote:
>>
>>> Looks like it is complaining that you are trying to load a 0.7 SSTable in
>>> 0.8.
>>>
>>>
>>> Cheers
>>>
>>>  -----------------
>>> Aaron Morton
>>> Freelance Cassandra Developer
>>> @aaronmorton
>>> http://www.thelastpickle.com
>>>
>>> On 23/09/2011, at 5:23 PM, Yan Chunlu wrote:
>>>
>>> sorry I did not look into it....  after check it I found version mismatch
>>> exception is in the log:
>>> ERROR [Thread-17] 2011-09-22 08:24:24,248 AbstractCassandraDaemon.java
>>> (line 139) Fatal exception in thread Thread[Thread-17,5,main]
>>> java.lang.RuntimeException: Cannot recover SSTable
>>> /disk2/cassandra/data/reddit/Comments-tmp-f-1 due to version mismatch.
>>> (current version is g).
>>>         at
>>> org.apache.cassandra.io.sstable.SSTableWriter.createBuilder(SSTableWriter.java:240)
>>>         at
>>> org.apache.cassandra.db.compaction.CompactionManager.submitSSTableBuild(CompactionManager.java:1097)
>>>         at
>>> org.apache.cassandra.streaming.StreamInSession.finished(StreamInSession.java:110)
>>>         at
>>> org.apache.cassandra.streaming.IncomingStreamReader.readFile(IncomingStreamReader.java:104)
>>>         at
>>> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:61)
>>>         at
>>> org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189)
>>>         at
>>> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117)
>>>
>>>
>>> does that mean I need to run scrub before running the loader?  could I
>>> just delete it and keep going?  thanks!
>>>
>>> On Fri, Sep 23, 2011 at 2:16 AM, Jonathan Ellis <jbellis@gmail.com>wrote:
>>>
>>>> Did you check for errors in logs on both loader + target?
>>>>
>>>> On Thu, Sep 22, 2011 at 10:52 AM, Yan Chunlu <springrider@gmail.com>
>>>> wrote:
>>>> > I took a snapshot of one of my node in a cluster 0.7.4(N=RF=3).   use
>>>> > sstableloader to load the snapshot data to another 1 node
>>>> cluster(N=RF=1).
>>>> >
>>>> > after execute  "bin/sstableloader  /disk2/mykeyspace/"
>>>> >
>>>> > it says"Starting client (and waiting 30 seconds for gossip) ..."
>>>> > "Streaming revelant part of  cf1.db..... to [10.23.2.4]"
>>>> > then showing the progress indicator and stopped. nothing changed after
>>>> > then.
>>>> > progress: [/10.28.53.16 1/72 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]]]
>>>> >
>>>> > I use nodetool to check the node 10.23.2.4, nothing changed. no data
>>>> copied
>>>> > to it. and the data dir also keep its original size. is there anything
>>>> > wrong? how can I tell what was going on there?
>>>> > thanks!
>>>>
>>>>
>>>>
>>>> --
>>>> Jonathan Ellis
>>>> Project Chair, Apache Cassandra
>>>> co-founder of DataStax, the source for professional Cassandra support
>>>> http://www.datastax.com
>>>>
>>>
>>>
>>>
>>
>>
>
>

Mime
View raw message