cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dathan Pattishall <datha...@gmail.com>
Subject Re: what causes MESSAGE-DESERIALIZER-POOL to spike
Date Mon, 26 Jul 2010 21:49:33 GMT
Okay so why would the pending be so large? I looked at iostat and the
diskload is not bad at all. The service times /etc are all good.
If it's not disk based I/O then is it thread thrashing? If so which thread
pool should I monitor more closely.


MESSAGE-DESERIALIZER-POOL:1

which would indicate there is one thread. If the design dictated one-thread
I would assume that its a service that should not have a pending queue.
Maybe there is a limitation in Cassandra

Name: MESSAGE-DESERIALIZER-POOL:1
State: WAITING on
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3c48e8f5
Total blocked: 10  Total waited: 5,427,548

Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
java.lang.Thread.run(Thread.java:619)



shows that there is a huge amount of waiting. Is there a tunable that I can
raise to remove this queue? Is it okay to have waited this high? If not what
is a bad threshold?


On Mon, Jul 26, 2010 at 1:31 PM, Peter Schuller <peter.schuller@infidyne.com
> wrote:

> > Hpw does one set/get the read-concurrency? Also which read-concurrency,
> for
> > which pool? According to the threads tab in jconsole read-concurrency
> could
> > be among different pools. There are row-read-stag threads, lots of write
> > threads, row mutation stage etc.
>
> Short version: Ignoremy concurrency speculation as it doesn't apply.
>
> Long version:
>
> I meant the ConcurrentReads settings in
> storage-conf.xml/cassandra.yaml (depending on version), but I was
> speculating.
>
> Having a look now, I presume it's not set to 1 because a minimum of 2
> seems to be required (DatabaseDescriptor.java). And since you ask
> where I suspect it's at its default value of 8 anyway.
>
> Also, looking at StageManager it appears this is applied to
> ROW-READ-STAGE so is indeed unrelated to MESSAGE-DESERIALIZER-POOL.
>
> MESSAGE-DESERIALIZER-POOL seems to be unrelated to disk I/O
> alltogether and seems to be used by MessagingService.java. So - please
> ignore my read concurrency speculation as it doesn't apply.
>
> --
> / Peter Schuller
>

Mime
View raw message