incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ingram Chen <ingramc...@gmail.com>
Subject Re: busy thread on IncomingStreamReader ?
Date Sun, 18 Apr 2010 01:47:23 GMT
FYI.

after upgrading jdk from  1.6.0_16 to  1.6.0_20, the problem solved.

On Fri, Apr 16, 2010 at 00:33, Ingram Chen <ingramchen@gmail.com> wrote:

> Hi all,
>
>  We setup two nodes and simply set replication factor=2 for test run.
>
> After both nodes, say, node A and node B, serve several hours, we found
> that "node A" always keep 300% cpu usage.
> (the other node is under 100% cpu, which is normal)
>
> thread dump on "node A" shows that there are 3 busy threads related to
> IncomingStreamReader:
>
> ==========================
>
> "Thread-66" prio=10 tid=0x00002aade4018800 nid=0x69e7 runnable
> [0x000000004030a000]
>    java.lang.Thread.State: RUNNABLE
>         at sun.misc.Unsafe.setMemory(Native Method)
>         at sun.nio.ch.Util.erase(Util.java:202)
>         at
> sun.nio.ch.FileChannelImpl.transferFromArbitraryChannel(FileChannelImpl.java:560)
>         at
> sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:603)
>         at
> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:62)
>         at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:66)
>
> "Thread-65" prio=10 tid=0x00002aade4017000 nid=0x69e6 runnable
> [0x000000004d44b000]
>    java.lang.Thread.State: RUNNABLE
>         at sun.misc.Unsafe.setMemory(Native Method)
>         at sun.nio.ch.Util.erase(Util.java:202)
>         at
> sun.nio.ch.FileChannelImpl.transferFromArbitraryChannel(FileChannelImpl.java:560)
>         at
> sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:603)
>         at
> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:62)
>         at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:66)
>
> "Thread-62" prio=10 tid=0x00002aade4014800 nid=0x4150 runnable
> [0x000000004d34a000]
>    java.lang.Thread.State: RUNNABLE
>         at sun.nio.ch.FileChannelImpl.size0(Native Method)
>         at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:309)
>         - locked <0x00002aaac450dcd0> (a java.lang.Object)
>         at
> sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:597)
>         at
> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:62)
>         at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:66)
>
> ===========================
>
> Is there anyone experience similar issue ?
>
> environments:
>
> OS   --- CentOS 5.4, Linux 2.6.18-164.15.1.el5 SMP x86_64 GNU/Linux
> Java --- build 1.6.0_16-b01, Java HotSpot(TM) 64-Bit Server VM (build
> 14.2-b01, mixed mode)
> Cassandra --- 0.6.0
> Node configuration --- node A and node B. both nodes use node A as Seed
> client --- Java thrift clients pick one node randomly to do read and write.
>
>
> --
> Ingram Chen
> online share order: http://dinbendon.net
> blog: http://www.javaworld.com.tw/roller/page/ingramchen
>



-- 
Ingram Chen
online share order: http://dinbendon.net
blog: http://www.javaworld.com.tw/roller/page/ingramchen

Mime
View raw message