cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Manley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9973) java.lang.IllegalStateException: Unable to compute when histogram overflowed
Date Sun, 23 Aug 2015 17:31:46 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708448#comment-14708448
] 

Mark Manley commented on CASSANDRA-9973:
----------------------------------------

Do we have an ETA for the release date of 2.2.1?  My ring was again crippled this morning
when several of my nodes spewed out hundreds of these errors a minute.  It corresponds to
the time that they stopped answering requests reliably.  If there is a workaround for 2.2.0,
I am all ears.

Thanks!

> java.lang.IllegalStateException: Unable to compute when histogram overflowed
> ----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-9973
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9973
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Mark Manley
>            Assignee: T Jake Luciani
>             Fix For: 2.2.1
>
>         Attachments: 9973.txt
>
>
> I recently, and probably mistakenly, upgraded one of my production C* clusters to 2.2.0.
 I am seeing these errors in the logs, followed by an intense period of garbage collection
until the node, then the ring, becomes crippled:
> {code}
> ERROR [OptionalTasks:1] 2015-08-04 03:24:56,057 CassandraDaemon.java:182 - Exception
in thread Thread[OptionalTasks:1,5,main]
> java.lang.IllegalStateException: Unable to compute when histogram overflowed
>         at org.apache.cassandra.utils.EstimatedHistogram.percentile(EstimatedHistogram.java:179)
~[apache-cassandra-2.2.0.jar:2.2.0]
>         at org.apache.cassandra.metrics.EstimatedHistogramReservoir$HistogramSnapshot.getValue(EstimatedHistogramReservoir.java:84)
~[apache-cassandra-2.2.0.jar:2.2.0]
>         at org.apache.cassandra.db.ColumnFamilyStore$3.run(ColumnFamilyStore.java:405)
~[apache-cassandra-2.2.0.jar:2.2.0]
>         at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
~[apache-cassandra-2.2.0.jar:2.2.0]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45]
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_45]
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}
> I am not sure if the GC instability is this or something else, but I though this histogram
overflow issue was fixed in 2.1.3?  Anyway, reporting now as a possible regression.  Please
let me know what I can provide in terms of information to help with this.  Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message