Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of jbellis@gmail.com designates
 74.125.82.172 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=N0du00fIPSzwhG6XwZRFpolSjm0B3Izrbcp4mNmlOmpPSYKT+Zk2SxoWIEg5EDRh87
         RkO59mcZecFItGur87SpuPpZQnj7t8TX+R7YFLYGcYtpGJ3ZTo/ir1C8ehRVoT2LLqr1
         XNGyW+YW3lR0+OHB9q+t/xYmr599qrN7CGlw4=
MIME-Version: 1.0
In-Reply-To: <AANLkTikEgmEtI-AVlmwnSpwih4wZ8lJ5hBSHKeeIYcEu@mail.gmail.com>
References: <AANLkTilIHB2S8lEK8r3Ml-co_7eil5bTVoGf5RE2S4v-@mail.gmail.com>
	<AANLkTiloMCvcp5JTQLKQ3AV7QA8Ln2Qa5Ao3_cRqo9rx@mail.gmail.com>
	<AANLkTimXEDb5YMdQ3MWzC8R6G6stQB-o-t742cDVIus5@mail.gmail.com>
	<AANLkTik7n6UcYduK10KYvO_J3EOvEwd8_pZscwqmyr_C@mail.gmail.com>
	<AANLkTikEgmEtI-AVlmwnSpwih4wZ8lJ5hBSHKeeIYcEu@mail.gmail.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Mon, 17 May 2010 11:09:45 -0500
Message-ID: <AANLkTilW3bKBPuPZxrmArf8RudJaKU3-Wk-ZdqymGmlf@mail.gmail.com>
Subject: Re: Load Balancing Mapper Tasks
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

That means they are blocking for something to be added to the task queue

On Mon, May 17, 2010 at 9:42 AM, Joost Ouwerkerk <joost@openplaces.org> wro=
te:
> At any given moment at least half of those threads are in the following
> state; what does it represent?
> Name: ROW-READ-STAGE:6
> State: WAITING on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@fea=
6030
> Total blocked: 44 =A0Total waited: 479
> Stack trace:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa=
it(AbstractQueuedSynchronizer.java:1925)
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:35=
8)
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:9=
47)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav=
a:907)
> java.lang.Thread.run(Thread.java:619)
>
> On Mon, May 17, 2010 at 12:53 AM, Jonathan Ellis <jbellis@gmail.com> wrot=
e:
>>
>> On Sun, May 16, 2010 at 2:52 PM, Joost Ouwerkerk <joost@openplaces.org>
>> wrote:
>> > Meanwhile. I'm still getting TimedOutException errors when mapping thi=
s
>> > 30-million row table, even when retrieving no data at all.=A0 It looks
>> > like it
>> > is related to disk activity on "hot" nodes (when the same cassandra no=
de
>> > has
>> > to handle many concurrent requests for adjacent range slices).=A0 Usin=
g
>> > 0.7
>> > trunk branch doesn't appear to alleviate it.=A0 CPU load is at about 2=
5%
>> > when
>> > this happens.=A0 Is there some kind of synchronization that might prev=
ent
>> > the
>> > same file from being scanned by multiple threads?
>>
>> No, but there's only `concurrent_reads` threads serving reads and if
>> you have a lot of ops in the queue ahead of you then that would do it.
>> =A0(pending tasks in row stage under cassandra.concurrent jmx.)
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>
>


--=20
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com