incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viktor Jevdokimov <Viktor.Jevdoki...@adform.com>
Subject RE: sstableloader throughput
Date Wed, 10 Apr 2013 15:39:08 GMT
Thanks, that could help to consider migration of the cluster to a newer version. We'll check
the difference.


From: Edward Capriolo [mailto:edlinuxguru@gmail.com] 
Sent: Wednesday, April 10, 2013 18:03
To: user@cassandra.apache.org
Subject: Re: sstableloader throughput

Yes. I did confirm that the 1.1 sstable loader works much better then the 1.0 version. The
changes were not easy to backport to the 1.0.X branch so it did not happen. It is likely that
1.2 is even better :)

On Wed, Apr 10, 2013 at 10:38 AM, Viktor Jevdokimov <Viktor.Jevdokimov@adform.com> wrote:
Rsync is not for our case.

Is sstableloader for 1.2.x faster?



From: Edward Capriolo [mailto:edlinuxguru@gmail.com]
Sent: Wednesday, April 10, 2013 15:52
To: user@cassandra.apache.org
Subject: Re: sstableloader throughput

Stables loader was slow in 1:0:x I had better luck with rsync. It was not fixed in the 1.0.x
series.

On Wednesday, April 10, 2013, Viktor Jevdokimov <Viktor.Jevdokimov@adform.com> wrote:
> Found https://issues.apache.org/jira/browse/CASSANDRA-3668
>
> Weird.
>
>
>
> Hi,
>
>
>
> We're using Casandra 1.0.12 sstableloader to import data from dedicated machine located
in DC1 into the cluster of 32 nodes (RF=4), 16 nodes in DC1 and 16 nodes in DC2.
>
>
>
> To disable throttle for sstableloader we set in casasndra.yaml:
>
> stream_throughput_outbound_megabits_per_sec: 0
>
>
>
> Outgoing network throughput is about 1Gbit, file copy from dedicated sstableloader machine
throughput is >90MB/s into DC1 node and >50MB/s into DC2 node.
>
>
>
> But with sstableloader outgoing network traffic is only 9MB/s, importing 1 sstable of
480MB into cluster (~60MB/node) takes 8-9 minutes. Even with 50MB/s import should take less
than a minute.
>
>
>
> Why sstableloader throughput is so low/slow?
>

Best regards / Pagarbiai

Viktor Jevdokimov
Senior Developer

Email: Viktor.Jevdokimov@adform.com
Phone: +370 5 212 3063
Fax: +370 5 261 0453

J. Jasinskio 16C,
LT-01112 Vilnius,
Lithuania



Disclaimer: The information contained in this message and attachments is intended solely for
the attention and use of the named addressee and may be confidential. If you are not the intended
recipient, you are reminded that the information remains the property of the sender. You must
not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this
message in error, please contact the sender immediately and irrevocably delete this message
and any copies.


Mime
View raw message