incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor <i...@4friends.od.ua>
Subject Re: size tiered compaction - improvement
Date Wed, 04 Apr 2012 04:52:51 GMT
Here is small python script I run once per day. You have to adjust size 
and/or age limits in the 'if' operator. Also I use mx4j interface for 
jmx calls.

#!/usr/bin/env python

import sys,os,glob,time,urllib2

CASSANDRA_DATA='/spool1/cassandra/data'
DONTTOUCH=('system',)

now = time.time()

def main():
     kss=[ks for ks in os.listdir(CASSANDRA_DATA) if ks not in DONTTOUCH]
     for ks in kss:
        sstables=[sst for sst in 
glob.glob(CASSANDRA_DATA+'/'+ks+'/'+'*-Data.db') if sst.find('-tmp-')==-1]
        for table in sstables:
            st = os.stat(table)
            age=(now-st.st_mtime)/24/3600
            size=st.st_size/1024/1024/1024
            if (age >= 5 and size >= 5) or age >= 10:
                 table_name = table.split('/')[-1]
                 print "compacting ", ks, table_name
                 
url='http://localhost:8081/invoke?operation=forceUserDefinedCompaction&objectname=org.apache.cassandra.db%%3Atype%%3DCompactionManager&value0=%s&type0=java.lang.String&value1=%s&type1=java.lang.String'%(ks,

table_name)
                 r=urllib2.urlopen(url)
                 time.sleep(1)

if __name__=='__main__':
     main()



On 04/04/2012 07:47 AM, igor@4friends.od.ua wrote:
>
> The first is keyspace name, second is sstable name (like 
> transaction-hc-1024-Data.db
>
>
>
> -----Original Message-----
> From: Radim Kolar <hsn@filez.com>
> To: user@cassandra.apache.org
> Sent: Wed, 04 Apr 2012 3:14
> Subject: Re: size tiered compaction - improvement
>
> Dne 3.4.2012 23 <tel:34201223>:04, igor@4friends.od.ua 
> <mailto:igor@4friends.od.ua> napsal(a):
> >
> > if you know for sure that you will free lot of space compacting some
> > old table, then you can call UserdefinedCompaction for this table(you
> > can do this from cron). There is also a ticket in jira with discussion
> > on per-sstable expierd column and tombstones counters.
> >
> you are talking about CompactionManager,forceUserDefinedCompaction
> mbean? it takes 2 argumenents, no description on them. i never got this
> work. NoSuchElementException returned
>


Mime
View raw message