cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Osman YOZGATLIOGLU <osman.yozgatlio...@krontech.com>
Subject Re: [Marketing Mail] Re: [Marketing Mail] Re: sstableloader question
Date Wed, 12 Oct 2016 16:35:26 GMT
Hello,

It's about 2500 sstables worth 25TB of data.
-t parameter doesn't change -t 1000 and -t 10000
Most probably I face some limitation at target cluster.
I'm preparing to split sstables and run up to ten parallel sstableloader sessions.

Regards,
Osman

On 11-10-2016 21:46, Rajath Subramanyam wrote:
How many sstables are you trying to load ? Running sstableloaders in parallel will help. Did
you try setting the "-t" parameter and see if you are getting the expected throughput ?

- Rajath

------------------------
Rajath Subramanyam


On Mon, Oct 10, 2016 at 2:02 PM, Osman YOZGATLIOGLU <osman.yozgatlioglu@krontech.com<mailto:osman.yozgatlioglu@krontech.com>>
wrote:
Hello,

Thank you Adam and Rajath.

I'll split input sstables and run parallel jobs for each.
I tested this approach and run 3 parallel sstableloader job without -t parameter.
I raised stream_throughput_outbound_megabits_per_sec parameter from 200 to 600 Mbit/sec at
all of target nodes.
But each job runs about 10MB/sec only and generates about 100Mbit'sec network traffic.
At total this can be much more. Source and target servers has plenty of unused cpu, io and
network resource.
Do you have any idea how can I increase speed of sstableloader job?

Regards,
Osman

On 10-10-2016 22:05, Rajath Subramanyam wrote:
Hi Osman,

You cannot restart the streaming only to the failed nodes specifically. You can restart the
sstableloader job itself. Compaction will eventually take care of the redundant rows.

- Rajath

------------------------
Rajath Subramanyam


On Sun, Oct 9, 2016 at 7:38 PM, Adam Hutson <adam@datascale.io<mailto:adam@datascale.io><mailto:adam@datascale.io<mailto:adam@datascale.io>>>
wrote:
It'll start over from the beginning.


On Sunday, October 9, 2016, Osman YOZGATLIOGLU <osman.yozgatlioglu@krontech.com<mailto:osman.yozgatlioglu@krontech.com><mailto:osman.yozgatlioglu@krontech.com<mailto:osman.yozgatlioglu@krontech.com>>>
wrote:
Hello,

I have running a sstableloader job.
Unfortunately some of nodes restarted since beginnig streaming.
I see streaming stop for those nodes.
Can I restart those streaming somehow?
Or if I restart sstableloader job, will it start from beginning?

Regards,
Osman


This e-mail message, including any attachments, is for the sole use of the person to whom
it has been sent, and may contain information that is confidential or legally protected. If
you are not the intended recipient or have received this message in error, you are not authorized
to copy, distribute, or otherwise use this message or its attachments. Please notify the sender
immediately by return e-mail and permanently delete this message and any attachments. KRON
makes no warranty that this e-mail is error or virus free.


--

Adam Hutson
Data Architect | DataScale
+1 (417) 224-5212<tel:%2B1%20%28417%29%20224-5212><tel:%2B1%20%28417%29%20224-5212>
adam@datascale.io<mailto:adam@datascale.io><mailto:adam@datascale.io<mailto:adam@datascale.io>>




This e-mail message, including any attachments, is for the sole use of the person to whom
it has been sent, and may contain information that is confidential or legally protected. If
you are not the intended recipient or have received this message in error, you are not authorized
to copy, distribute, or otherwise use this message or its attachments. Please notify the sender
immediately by return e-mail and permanently delete this message and any attachments. KRON
makes no warranty that this e-mail is error or virus free.




This e-mail message, including any attachments, is for the sole use of the person to whom
it has been sent, and may contain information that is confidential or legally protected. If
you are not the intended recipient or have received this message in error, you are not authorized
to copy, distribute, or otherwise use this message or its attachments. Please notify the sender
immediately by return e-mail and permanently delete this message and any attachments. KRON
makes no warranty that this e-mail is error or virus free.

Mime
View raw message