cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Guyomar <nicolas.guyo...@gmail.com>
Subject Re: old big tombstone data file occupy much disk space
Date Fri, 01 Sep 2017 09:36:32 GMT
Whoops sorry I mislead you with cassandra 2.1 behavior, you were right
giving your sstable full path , what kind of log do you have when you
trigger the compaction with the full path ?

On 1 September 2017 at 11:30, Nicolas Guyomar <nicolas.guyomar@gmail.com>
wrote:

> Well, not sure why you reached a memory usage limit, but according to the
> 3.0 branche's code : https://github.com/apache/
> cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/
> CompactionManager.java#L632 you just need to give the sstable filename,
> and Cassandra manage to find it based on cassandra version, sstable
> filename convention and so on
>
> Are you sure those sstables you are trying to get rid off are really in an
> active schema, and not some leftover from an old keyspace/table? This is
> what "schema does not exist" means to me.
>
> On 1 September 2017 at 11:06, qf zhou <zhouqf2013@gmail.com> wrote:
>
>> I  found the  following log.  What does it mean ?
>>
>> INFO  [CompactionExecutor:11] 2017-09-01 16:55:47,909
>> NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot
>> allocate chunk of 1.000MiB
>> WARN  [RMI TCP Connection(1714)-127.0.0.1] 2017-09-01 17:02:42,516
>> CompactionManager.java:704 - Schema does not exist for file
>> mc-151276-big-Data.db. Skipping.
>>
>>
>> 在 2017年9月1日,下午4:54,Nicolas Guyomar <nicolas.guyomar@gmail.com>
写道:
>>
>> You should have a log coming from the CompactionManager (in cassandra
>> system.log) when you try the command, what does it says  ?
>>
>> On 1 September 2017 at 10:07, qf zhou <zhouqf2013@gmail.com> wrote:
>>
>>> When I run the command,  the following occurs and  it returns null.
>>>
>>> Is it normal ?
>>>
>>> echo "run -b org.apache.cassandra.db:type=CompactionManager
>>> forceUserDefinedCompaction mc-100963-big-Data.db" | java -jar
>>> /opt/cassandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar   -l
>>> localhost:7199
>>>
>>>
>>> Welcome to JMX terminal. Type "help" for available commands.
>>> $>run -b org.apache.cassandra.db:type=CompactionManager
>>> forceUserDefinedCompaction mc-100963-big-Data.db
>>> #calling operation forceUserDefinedCompaction of mbean
>>> org.apache.cassandra.db:type=CompactionManager
>>> #operation returns:
>>> null
>>>
>>>
>>>
>>>
>>> 在 2017年9月1日,下午3:49,Nicolas Guyomar <nicolas.guyomar@gmail.com>
写道:
>>>
>>> Hi,
>>>
>>> Last time I used forceUserDefinedCompaction, I got myself a headache
>>> because I was trying to use a full path like you're doing, but in fact it
>>> just need the sstable as parameter
>>>
>>> Can you just try :
>>>
>>> echo "run -b org.apache.cassandra.db:type=CompactionManager
>>> forceUserDefinedCompaction mc-100963-big-Data.db" | java -jar
>>> /opt/cassandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar   -l
>>> localhost:7199
>>>
>>>
>>>
>>> On 1 September 2017 at 08:29, qf zhou <zhouqf2013@gmail.com> wrote:
>>>
>>>>
>>>> dataPath=/hdd3/cassandra/data/gps/gpsfullwithstate-073e51a0c
>>>> db811e68dce511be6a305f6/mc-100963-big-Data.db
>>>> echo "run -b org.apache.cassandra.db:type=CompactionManager
>>>> forceUserDefinedCompaction $dataPath" | java -jar
>>>> /opt/cassandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar   -l
>>>> localhost:7199
>>>>
>>>> In the above, I am using a jmx method. But it seems that the file size
>>>> doesn’t change. My command is wrong ?
>>>>
>>>> > 在 2017年9月1日,下午2:17,Jeff Jirsa <jjirsa@gmail.com>
写道:
>>>> >
>>>> > User defined compaction to do a single sstable compaction on just
>>>> that sstable
>>>> >
>>>> > It's a nodetool command in very recent versions, or a jmx method in
>>>> older versions
>>>> >
>>>> >
>>>> > --
>>>> > Jeff Jirsa
>>>> >
>>>> >
>>>> >> On Aug 31, 2017, at 11:04 PM, qf zhou <zhouqf2013@gmail.com>
wrote:
>>>> >>
>>>> >> I am using  a cluster with  3 nodes and  the cassandra version is
>>>> 3.0.9. I have used it about 6 months. Now each node has about 1.5T data in
>>>> the disk.
>>>> >> I found some sstables file are over 300G. Using the  sstablemetadata
>>>> command,  I found it:  Estimated droppable tombstones: 0.9622972799707109.
>>>> >> It is obvious that too much tombstone data exists.
>>>> >> The default_time_to_live = 8640000(100 days) and   gc_grace_seconds
>>>> = 432000(5 days).  Using nodetool  compactionstats, I found the some
>>>> compaction processes exists.
>>>> >> So I really  want to know how to clear tombstone data ?  otherwise
>>>> the disk space will cost too much.
>>>> >> I really need some help, because some few people know cassandra
in
>>>> my company.
>>>> >> Thank you very much!
>>>> >>
>>>> >>
>>>> >> ------------------------------------------------------------
>>>> ---------
>>>> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>>> >> For additional commands, e-mail: user-help@cassandra.apache.org
>>>> >>
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>>> > For additional commands, e-mail: user-help@cassandra.apache.org
>>>> >
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>>>> For additional commands, e-mail: user-help@cassandra.apache.org
>>>>
>>>>
>>>
>>>
>>
>>
>

Mime
View raw message