On Mon, Mar 21, 2016 at 2:15 PM, Alain RODRIGUEZ <arodrime@gmail.com> wrote:

What hardware do you use? Can you see it running at the limits (CPU / disks IO)? Is there any error on system logs, are disks doing fine?

Nodes are c3.2xlarge instances on AWS. The nodes are relatively idle, and, as said in the original email, the other nodes are handling the same load and are doing just fine (no warnings or errors in the system and cassandra logs). I mean, even this node is doing very fine, it seems as if there is some internal bug in the way pending tasks are counted.
How is configured the concurrent_compactors setting? It looks like you are using 2 from 'nodetool compactionstats' output. Could you increase that or is it the number of cores of the machine?

It's 2. I can certainly increase it, but, as I proved earlier, it doesn't do anything, because if I do a "nodetool drain" and wait a while, the compactions completely stop (and the node goes totally idle), despite the pending compactions remaining high, so this really doesn't feel like a problem of Cassandra not being able to keep up with the compactions.