incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard grossman <richie...@gmail.com>
Subject Re: Timeout Exception
Date Tue, 10 Nov 2009 08:28:39 GMT
For me the timeout occurs when I've a lot of request to the server. It was
working at 100% CPU and it seems that cassandra was receiving request and
keep them into the queue for more than 5000 millis then all the request was
rejected with timeout.

I can't think about another solution or to increase the timeout (not
ideally) or add more cassandra instance to answer high requests number
faster.


On Tue, Nov 10, 2009 at 7:22 AM, Jonathan Ellis <jbellis@gmail.com> wrote:

> What's causing the timeout?  An error on the source node, or just
> slowness?  If the latter, how many rows are in your multiget?
>
> On Mon, Nov 9, 2009 at 10:25 PM, Chris Were <chris.were@gmail.com> wrote:
> >
> > I'm getting a Timeout Exception every now and again (currently every
> couple
> > of minutes or so).
> > Using revision 833288. Quorum set to ONE. My cassandra instance has been
> > running for two days and the data directory is around 16GB. I'm not sure
> > what the problem is, but let me know of any tests I can do to help reduce
> > the problem further. There are two variations on the exception, I have
> > pasted them both below.
> > ERROR [pool-1-thread-63] 2009-11-09 20:17:27,579 Cassandra.java (line
> > org.apache.cassandra.service.Cassandra$Processor) Internal error
> processing
> > get_slice
> > java.lang.RuntimeException: java.util.concurrent.TimeoutException:
> Operation
> > timed out - received only 0 responses from  .
> > at
> >
> org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:103)
> > at
> >
> org.apache.cassandra.service.CassandraServer.getSlice(CassandraServer.java:177)
> > at
> >
> org.apache.cassandra.service.CassandraServer.multigetSliceInternal(CassandraServer.java:252)
> > at
> >
> org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:215)
> > at
> >
> org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:668)
> > at
> >
> org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:624)
> > at
> >
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:636)
> > Caused by: java.util.concurrent.TimeoutException: Operation timed out -
> > received only 0 responses from  .
> > at
> >
> org.apache.cassandra.service.QuorumResponseHandler.get(QuorumResponseHandler.java:79)
> > at
> >
> org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:408)
> > at
> >
> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:333)
> > at
> >
> org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:95)
> > ... 9 more
> > ERROR [pool-1-thread-19] 2009-11-09 11:29:18,731 Cassandra.java (line
> > org.apache.cassandra.service.Cassandra$Processor) Internal error
> processing
> > get_slice
> > java.lang.RuntimeException: java.util.concurrent.TimeoutException:
> Operation
> > timed out - received only 1 responses from /10.121.217.5 .
> >         at
> >
> org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:103)
> >         at
> >
> org.apache.cassandra.service.CassandraServer.getSlice(CassandraServer.java:177)
> >         at
> >
> org.apache.cassandra.service.CassandraServer.multigetSliceInternal(CassandraServer.java:252)
> >         at
> >
> org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:215)
> >         at
> >
> org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:668)
> >         at
> >
> org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:624)
> >         at
> >
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:253)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >         at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >         at java.lang.Thread.run(Thread.java:636)
> > Caused by: java.util.concurrent.TimeoutException: Operation timed out -
> > received only 1 responses from /10.121.217.5 .
> >         at
> >
> org.apache.cassandra.service.QuorumResponseHandler.get(QuorumResponseHandler.java:79)
> >         at
> >
> org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:408)
> >         at
> >
> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:333)
> >         at
> >
> org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:95)
> >         ... 9 more
> > Cheers,
> > Chris
>

Mime
View raw message