How large are your SSTables on disk? My thought was because you have so many on disk, we have to store the bloom filter + every 128 keys from index in memory.
But as I understand
store in memory happens only when read happens, i do only inserts. And i
think that memory doesn't problem, because heap allocations looks like
saw (in max Heap allocations get about 5,5 GB then they reduce to 2GB)
Also when i increase Heap Size to 7GB, situation stay mach better, but nodes frozen still happens, and in gc.log I steel see:
Total time for which application threads were stopped: 20.0686307 seconds
lines (right not so often, like before)