incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Manuel Formoso <jform...@gmail.com>
Subject Re: I don't understand shuffle progress
Date Wed, 18 Sep 2013 03:05:50 GMT
I have been trying to make it work non-stop since Friday afternoon. I
officially gave up today and I'm going to go the sstableloader route.

I wrote a little of what I tried here:
http://seniorgeek.com.ar/blog/2013/09/16/tips-for-running-cassandra-shuffle/
(I have yet to update it with the fact that I had to give up)

I would strongly recommend you don't use shuffle unless you have very
little data to move around.


On Tue, Sep 17, 2013 at 10:41 PM, Paulo Motta <pauloricardomg@gmail.com>wrote:

> That is very disappointing to hear. Vnodes support is one of the main
> reasons we're upgrading from 1.1.X to 1.2.X.
>
> So you're saying the only feasible way of enabling VNodes on an upgraded C*
> 1.2 is by doing fork writes to a brand new cluster + bulk load of sstables
> from the old cluster? Or is it possible to succeed on shuffling, even if
> that means waiting some weeks for the shuffle to complete?
>
>
> 2013/9/17 Robert Coli <rcoli@eventbrite.com>
>
> > On Tue, Sep 17, 2013 at 4:00 PM, Juan Manuel Formoso <jformoso@gmail.com
> > >wrote:
> >
> > > Any better alternatives than creating a small application that reads
> from
> > > one cluster and inserts in the new one that anybody can suggest?
> > >
> > >
> > http://www.palominodb.com/blog/2012/09/25/bulk-loading-options-cassandra
> >
> > In theory if you wanted to do the "copy-the-files" method while enabling
> > vnodes on the target cluster, you could :
> >
> > 1) create new target cluster with vnodes enabled
> > 2) fork writes so they go to both source and target cluster
> > 3) copy 100% of sstables from all source nodes to all target nodes (being
> > sure to ensure non-collision of sstables of names, probably by adding a
> few
> > hundreds/thousands to the sequence of various nodes in a predictable
> > fashion)
> > 4) be certain that you did not accidentally resurrect data from purged
> > source sstables in 3)
> > 5) run cleanup compaction on all nodes in target cluster
> > 6) turn off writes to old source cluster
> >
> > =Rob
> > * notes that this process would make a good blog post.. :D
> >
>
>
>
> --
> Paulo Ricardo
>
> --
> European Master in Distributed Computing***
> Royal Institute of Technology - KTH
> *
> *Instituto Superior T├ęcnico - IST*
> *http://paulormg.com*
>



-- 
*Juan Manuel Formoso
*Senior Geek
http://twitter.com/juanformoso
http://seniorgeek.com.ar
LLAP

Mime
View raw message