incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean" <Dean.Hil...@nrel.gov>
Subject Re: Are Writes disk-bound rather than CPU-bound?
Date Tue, 23 Jul 2013 18:15:39 GMT
Also, thinking about it more…..

There are constant write loads where if the cluster is big enough compared
to the write load that the flushes are not happening that often causing a
profile like that to be cpu bound.

In my mind it really is about what configuration you have.  If I have a
high write load and I increase my cluster size 10 fold, it may be the case
that I am limited by CPU as the memtable flushes are not happening as
often since writes are thinned out across the cluster.

Dean

On 7/23/13 12:12 PM, "Hiller, Dean" <Dean.Hiller@nrel.gov> wrote:

>Out of curiosity, isn't what is really happening is this
>
>"As writes keep coming in, memory fills up causing flushes to the commit
>log disk of the whole memtable.  In a bursting scenario, writes are thus
>limited only by memory and cpu in short bursting cases that tend to fit
>in memory.  In a more long window of constant writes, the writes become
>limited by the constant flushing to disk of the memtable"
>
>Ie. We kind of have two scenarios in my mind if I got that right that is
>from my understanding that cassandra is write to memory first and
>asynchronously write to disk.
>
>Later,
>Dean
>
>From: Alex Popescu <alexp@datastax.com<mailto:alexp@datastax.com>>
>Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>"
><user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
>Date: Tuesday, July 23, 2013 12:07 PM
>To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>"
><user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
>Cc: 
>"cassandra-user@incubator.apache.org<mailto:cassandra-user@incubator.apach
>e.org>" 
><cassandra-user@incubator.apache.org<mailto:cassandra-user@incubator.apach
>e.org>>
>Subject: Re: Are Writes disk-bound rather than CPU-bound?
>
>I see pretty much the same formulation in the 1.2 docs, so I'm wondering
>what would be the best rewrite of that paragraph?
>
>
>On Tue, Jul 23, 2013 at 9:00 AM, hajjat
><hajjat@purdue.edu<mailto:hajjat@purdue.edu>> wrote:
>
>
>On Tue, Jul 23, 2013 at 5:05 AM, aaron morton [via [hidden
>email]<http://user/SendEmail.jtp?type=node&node=7589248&i=0>] <[hidden
>email]<http://user/SendEmail.jtp?type=node&node=7589248&i=1>> wrote:
>“/Insert-heavy workloads will actually be CPU-bound in Cassandra before
>being
>memory-bound/”
>What is the source for that ?
>
>http://www.datastax.com/docs/0.8/cluster_architecture/cluster_planning
>​​
>
>This is because everything is
>*first *written to the commit log *on disk*. Any thoughts??
>Pretty much.
>
>Cheers
>
>-----------------
>Aaron Morton
>Cassandra Consultant
>New Zealand
>
>@aaronmorton
>http://www.thelastpickle.com
>
>On 22/07/2013, at 8:58 AM, hajjat <[hidden
>email]<http://user/SendEmail.jtp?type=node&node=7589236&i=0>> wrote:
>
>“/Insert-heavy workloads will actually be CPU-bound in Cassandra before
>being
>memory-bound/”
>
>However, from reading the documentation
>(http://www.datastax.com/docs/1.2/dml/about_writes) it seems the disk is
>the
>real bottleneck in Writes rather than the CPU. This is because everything
>is
>*first *written to the commit log *on disk*. Any thoughts??
>
>
>
>--
>View this message in context:
>http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Are-Write
>s-disk-bound-rather-than-CPU-bound-tp7589191.html
>Sent from the [hidden
>email]<http://user/SendEmail.jtp?type=node&node=7589236&i=1> mailing list
>archive at Nabble.com<http://Nabble.com>.
>
>
>
>________________________________
>If you reply to this email, your message will be added to the discussion
>below:
>http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Are-Write
>s-disk-bound-rather-than-CPU-bound-tp7589191p7589236.html
>To unsubscribe from Are Writes disk-bound rather than CPU-bound?, click
>here.
>NAML<http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/temp
>late/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.n
>aml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.Na
>bbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_su
>bscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_
>instant_email%21nabble%3Aemail.naml>
>
>
>
>--
>Mohammad Hajjat
>Ph.D. Student
>Electrical and Computer Engineering
>Purdue University
>
>________________________________
>View this message in context: Re: Are Writes disk-bound rather than
>CPU-bound?<http://cassandra-user-incubator-apache-org.3065146.n2.nabble.co
>m/Are-Writes-disk-bound-rather-than-CPU-bound-tp7589191p7589248.html>
>
>Sent from the cassandra-user@incubator.apache.org mailing list
>archive<http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/>
> at Nabble.com.
>
>
>
>--
>
>:- a)
>
>
>Alex Popescu
>@al3xandru

Mime
View raw message