cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: nodetool cfstats on 1.0.0-rc1 throws an exception
Date Tue, 04 Oct 2011 08:20:46 GMT
That row has a size of 819 peta bytes, so something is odd there. The error is a result of
that value been so huge. When you rant he same script on 0.8.6 what was the max size of the
Migrations CF ?

As Jonathan says, it's unlikely anyone would have tested creating 5000 CF's. Most people only
create a few 10's of CF's at most.

either use fewer CF's or…

* dump the Migrations CF using sstable2json to take a look around 
* work out steps to reproduce and report it on Jira

Hope that helps. 

-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 4/10/2011, at 11:30 AM, Ramesh Natarajan wrote:

> We recreated the schema using the same input file on both clusters and they are running
identical load.  
> 
> Isn't the exception thrown in the system CF?
> 
> this line looks strange:
> 
> Compacted row maximum size: 9223372036854775807
> 
> thanks
> Ramesh
> 
> On Mon, Oct 3, 2011 at 5:26 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
> Looks like you have unexpectedly large rows in your 1.0 cluster but
> not 0.8.  I guess you could use sstable2json to manually check your
> row sizes.
> 
> On Mon, Oct 3, 2011 at 5:20 PM, Ramesh Natarajan <ramesh25@gmail.com> wrote:
> > It happens all the time on 1.0. It doesn't happen on 0.8.6.  Is there any
> > thing I can do to check?
> > thanks
> > Ramesh
> >
> > On Mon, Oct 3, 2011 at 5:15 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
> >>
> >> My suspicion would be that it has more to do with "rare case when
> >> running with 5000 CFs" than "1.0 regression."
> >>
> >> On Mon, Oct 3, 2011 at 5:00 PM, Ramesh Natarajan <ramesh25@gmail.com>
> >> wrote:
> >> > We have about 5000 column family and when we run the nodetool cfstats it
> >> > throws out this exception...  this is running 1.0.0-rc1
> >> > This seems to work on 0.8.6.  Is this a bug in 1.0.0?
> >> >
> >> > thanks
> >> > Ramesh
> >> > Keyspace: system
> >> >         Read Count: 28
> >> >         Read Latency: 5.8675 ms.
> >> >         Write Count: 3
> >> >         Write Latency: 0.166 ms.
> >> >         Pending Tasks: 0
> >> >                 Column Family: Schema
> >> >                 SSTable count: 4
> >> >                 Space used (live): 4293758276
> >> >                 Space used (total): 4293758276
> >> >                 Number of Keys (estimate): 5376
> >> >                 Memtable Columns Count: 0
> >> >                 Memtable Data Size: 0
> >> >                 Memtable Switch Count: 0
> >> >                 Read Count: 3
> >> >                 Read Latency: NaN ms.
> >> >                 Write Count: 0
> >> >                 Write Latency: NaN ms.
> >> >                 Pending Tasks: 0
> >> >                 Key cache capacity: 53
> >> >                 Key cache size: 2
> >> >                 Key cache hit rate: NaN
> >> >                 Row cache: disabled
> >> >                 Compacted row minimum size: 104
> >> >                 Compacted row maximum size: 1955666
> >> >                 Compacted row mean size: 1508515
> >> >                 Column Family: HintsColumnFamily
> >> >                 SSTable count: 0
> >> >                 Space used (live): 0
> >> >                 Space used (total): 0
> >> >                 Number of Keys (estimate): 0
> >> >                 Memtable Columns Count: 0
> >> >                 Memtable Data Size: 0
> >> >                 Memtable Switch Count: 0
> >> >                 Read Count: 5
> >> >                 Read Latency: NaN ms.
> >> >                 Write Count: 0
> >> >                 Write Latency: NaN ms.
> >> >                 Pending Tasks: 0
> >> >                 Key cache capacity: 1
> >> >                 Key cache size: 0
> >> >                 Key cache hit rate: NaN
> >> >                 Row cache: disabled
> >> >                 Compacted row minimum size: 0
> >> >                 Compacted row maximum size: 0
> >> >                 Compacted row mean size: 0
> >> >                 Column Family: LocationInfo
> >> >                 SSTable count: 1
> >> >                 Space used (live): 6947
> >> >                 Space used (total): 6947
> >> >                 Number of Keys (estimate): 128
> >> >                 Memtable Columns Count: 0
> >> >                 Memtable Data Size: 0
> >> >                 Memtable Switch Count: 2
> >> >                 Read Count: 20
> >> >                 Read Latency: NaN ms.
> >> >                 Write Count: 3
> >> >                 Write Latency: NaN ms.
> >> >                 Pending Tasks: 0
> >> >                 Key cache capacity: 1
> >> >                 Key cache size: 1
> >> >                 Key cache hit rate: NaN
> >> >                 Row cache: disabled
> >> >                 Compacted row minimum size: 73
> >> >                 Compacted row maximum size: 258
> >> >                 Compacted row mean size: 185
> >> >                 Column Family: Migrations
> >> >                 SSTable count: 4
> >> >                 Space used (live): 4315909643
> >> >                 Space used (total): 4315909643
> >> >                 Number of Keys (estimate): 512
> >> >                 Memtable Columns Count: 0
> >> >                 Memtable Data Size: 0
> >> >                 Memtable Switch Count: 0
> >> >                 Read Count: 0
> >> >                 Read Latency: NaN ms.
> >> >                 Write Count: 0
> >> >                 Write Latency: NaN ms.
> >> >                 Pending Tasks: 0
> >> >                 Key cache capacity: 5
> >> >                 Key cache size: 0
> >> >                 Key cache hit rate: NaN
> >> >                 Row cache: disabled
> >> >                 Compacted row minimum size: 5839589
> >> >                 Compacted row maximum size: 9223372036854775807
> >> > Exception in thread "main" java.lang.IllegalStateException: Unable to
> >> > compute ceiling for max when histogram overflowed
> >> >         at
> >> >
> >> > org.apache.cassandra.utils.EstimatedHistogram.mean(EstimatedHistogram.java:170)
> >> >         at
> >> > org.apache.cassandra.db.DataTracker.getMeanRowSize(DataTracker.java:395)
> >> >         at
> >> >
> >> > org.apache.cassandra.db.ColumnFamilyStore.getMeanRowSize(ColumnFamilyStore.java:275)
> >> >         at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
> >> >         at
> >> >
> >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> >         at java.lang.reflect.Method.invoke(Method.java:597)
> >> >         at
> >> >
> >> > com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
> >> >         at
> >> >
> >> > com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
> >> >         at
> >> >
> >> > com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
> >> >         at
> >> > com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:65)
> >> >         at
> >> > com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:216)
> >> >         at
> >> >
> >> > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666)
> >> >         at
> >> >
> >> > com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638)
> >> >         at
> >> >
> >> > javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1404)
> >> >         at
> >> >
> >> > javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
> >> >         at
> >> >
> >> > javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
> >> >         at
> >> >
> >> > javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
> >> >         at
> >> >
> >> > javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:600)
> >> >         at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
> >> >         at
> >> >
> >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> >         at java.lang.reflect.Method.invoke(Method.java:597)
> >> >         at
> >> > sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
> >> >         at sun.rmi.transport.Transport$1.run(Transport.java:159)
> >> >         at java.security.AccessController.doPrivileged(Native Method)
> >> >         at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> >> >         at
> >> > sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
> >> >         at
> >> >
> >> > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
> >> >         at
> >> >
> >> > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
> >> >         at
> >> >
> >> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >> >         at
> >> >
> >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >> >         at java.lang.Thread.run(Thread.java:662)
> >> >
> >>
> >>
> >>
> >> --
> >> Jonathan Ellis
> >> Project Chair, Apache Cassandra
> >> co-founder of DataStax, the source for professional Cassandra support
> >> http://www.datastax.com
> >
> >
> 
> 
> 
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
> 


Mime
View raw message