cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Guyomar <nicolas.guyo...@gmail.com>
Subject Re: old big tombstone data file occupy much disk space
Date Fri, 01 Sep 2017 09:30:17 GMT
Well, not sure why you reached a memory usage limit, but according to the
3.0 branche's code :
https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java#L632
you just need to give the sstable filename, and Cassandra manage to find it
based on cassandra version, sstable filename convention and so on

Are you sure those sstables you are trying to get rid off are really in an
active schema, and not some leftover from an old keyspace/table? This is
what "schema does not exist" means to me.

On 1 September 2017 at 11:06, qf zhou <zhouqf2013@gmail.com> wrote:

> I  found the  following log.  What does it mean ?
>
> INFO  [CompactionExecutor:11] 2017-09-01 16:55:47,909 NoSpamLogger.java:91
> - Maximum memory usage reached (512.000MiB), cannot allocate chunk of
> 1.000MiB
> WARN  [RMI TCP Connection(1714)-127.0.0.1] 2017-09-01 17:02:42,516
> CompactionManager.java:704 - Schema does not exist for file
> mc-151276-big-Data.db. Skipping.
>
>
> 在 2017年9月1日,下午4:54,Nicolas Guyomar <nicolas.guyomar@gmail.com>
写道:
>
> You should have a log coming from the CompactionManager (in cassandra
> system.log) when you try the command, what does it says  ?
>
> On 1 September 2017 at 10:07, qf zhou <zhouqf2013@gmail.com> wrote:
>
>> When I run the command,  the following occurs and  it returns null.
>>
>> Is it normal ?
>>
>> echo "run -b org.apache.cassandra.db:type=CompactionManager
>> forceUserDefinedCompaction mc-100963-big-Data.db" | java -jar
>> /opt/cassandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar   -l localhost:7199
>>
>>
>> Welcome to JMX terminal. Type "help" for available commands.
>> $>run -b org.apache.cassandra.db:type=CompactionManager
>> forceUserDefinedCompaction mc-100963-big-Data.db
>> #calling operation forceUserDefinedCompaction of mbean
>> org.apache.cassandra.db:type=CompactionManager
>> #operation returns:
>> null
>>
>>
>>
>>
>> 在 2017年9月1日,下午3:49,Nicolas Guyomar <nicolas.guyomar@gmail.com>
写道:
>>
>> Hi,
>>
>> Last time I used forceUserDefinedCompaction, I got myself a headache
>> because I was trying to use a full path like you're doing, but in fact it
>> just need the sstable as parameter
>>
>> Can you just try :
>>
>> echo "run -b org.apache.cassandra.db:type=CompactionManager
>> forceUserDefinedCompaction mc-100963-big-Data.db" | java -jar
>> /opt/cassandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar   -l localhost:7199
>>
>>
>>
>> On 1 September 2017 at 08:29, qf zhou <zhouqf2013@gmail.com> wrote:
>>
>>>
>>> dataPath=/hdd3/cassandra/data/gps/gpsfullwithstate-073e51a0c
>>> db811e68dce511be6a305f6/mc-100963-big-Data.db
>>> echo "run -b org.apache.cassandra.db:type=CompactionManager
>>> forceUserDefinedCompaction $dataPath" | java -jar
>>> /opt/cassandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar   -l
>>> localhost:7199
>>>
>>> In the above, I am using a jmx method. But it seems that the file size
>>> doesn’t change. My command is wrong ?
>>>
>>> > 在 2017年9月1日,下午2:17,Jeff Jirsa <jjirsa@gmail.com> 写道:
>>> >
>>> > User defined compaction to do a single sstable compaction on just that
>>> sstable
>>> >
>>> > It's a nodetool command in very recent versions, or a jmx method in
>>> older versions
>>> >
>>> >
>>> > --
>>> > Jeff Jirsa
>>> >
>>> >
>>> >> On Aug 31, 2017, at 11:04 PM, qf zhou <zhouqf2013@gmail.com> wrote:
>>> >>
>>> >> I am using  a cluster with  3 nodes and  the cassandra version is
>>> 3.0.9. I have used it about 6 months. Now each node has about 1.5T data in
>>> the disk.
>>> >> I found some sstables file are over 300G. Using the  sstablemetadata
>>> command,  I found it:  Estimated droppable tombstones: 0.9622972799707109.
>>> >> It is obvious that too much tombstone data exists.
>>> >> The default_time_to_live = 8640000(100 days) and   gc_grace_seconds
=
>>> 432000(5 days).  Using nodetool  compactionstats, I found the some
>>> compaction processes exists.
>>> >> So I really  want to know how to clear tombstone data ?  otherwise
>>> the disk space will cost too much.
>>> >> I really need some help, because some few people know cassandra in my
>>> company.
>>> >> Thank you very much!
>>> >>
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>> >> For additional commands, e-mail: user-help@cassandra.apache.org
>>> >>
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>> > For additional commands, e-mail: user-help@cassandra.apache.org
>>> >
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>> For additional commands, e-mail: user-help@cassandra.apache.org
>>>
>>>
>>
>>
>
>

Mime
View raw message