incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wayne <wav...@gmail.com>
Subject Re: Read Latency
Date Sat, 16 Oct 2010 20:37:44 GMT
ERROR [pool-1-thread-64] 2010-10-16 20:27:55,396 Cassandra.java (line 1280)
Internal error processing get_slice
java.lang.AssertionError
    at
org.apache.cassandra.service.ReadResponseResolver.resolve(ReadResponseResolver.java:88)
    at
org.apache.cassandra.service.ReadResponseResolver.resolve(ReadResponseResolver.java:44)
    at
org.apache.cassandra.service.QuorumResponseHandler.get(QuorumResponseHandler.java:86)
    at
org.apache.cassandra.service.StorageProxy.strongRead(StorageProxy.java:489)
    at
org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:353)
    at
org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:97)
    at
org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:180)
    at
org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:262)
    at
org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:223)
    at
org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
    at
org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
    at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)


On Sat, Oct 16, 2010 at 9:52 AM, Jonathan Ellis <jbellis@gmail.com> wrote:

> Stack trace from cassandra log?
>
> On Sat, Oct 16, 2010 at 6:50 AM, Wayne <wav100@gmail.com> wrote:
> > While doing a read I get a TApplicationException: Internal Error
> processing
> > get_slice.
> >
> > On Fri, Oct 15, 2010 at 5:49 PM, Jonathan Ellis <jbellis@gmail.com>
> wrote:
> >>
> >> On Fri, Oct 15, 2010 at 2:21 PM, Wayne <wav100@gmail.com> wrote:
> >> > The optimization definitely shaved off some time. Now it is running
> >> > about 3x
> >> > CFSTATS reported time. Below are the logs.
> >> >
> >> > There is a ~300ms time frame after the last ResponseVerbHandler prior
> to
> >> > the
> >> > resolver starting. Based on a quorum read the response resolver should
> >> > kick
> >> > after 2 reads come in correct? That would mean it waited 500ms before
> >> > starting. Where is this time going?
> >>
> >> It's going to deserialize the replies to see if it has both the data
> >> and enough digests to call resolve().  Then resolve deserializes them
> >> a 2nd time, so there's an easy win there caching the first
> >> deserialize, in the patch attached.  (Applies on top of the previous
> >> one, which has been committed to the 0.6 svn branch at
> >> https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6.)
> >>
> >> > There is still the 3+ second delay between the last 2 entries. Is this
> >> > the
> >> > thrift server?
> >>
> >> It's converting the reply from Cassandra's internal representation to
> >> thrift, and sending it to the client.  I suspect most of the time is
> >> the actual sending/waiting for the client to read part.  Patch 2 also
> >> includes a debug statement after the convert-to-thrift stage to
> >> verify.
> >>
> >> --
> >> Jonathan Ellis
> >> Project Chair, Apache Cassandra
> >> co-founder of Riptano, the source for professional Cassandra support
> >> http://riptano.com
> >
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>

Mime
View raw message