Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9B22C2004F5 for ; Fri, 1 Sep 2017 09:50:02 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9998216B2E0; Fri, 1 Sep 2017 07:50:02 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B9C1C16B2D4 for ; Fri, 1 Sep 2017 09:50:01 +0200 (CEST) Received: (qmail 44753 invoked by uid 500); 1 Sep 2017 07:49:59 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 44743 invoked by uid 99); 1 Sep 2017 07:49:59 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Sep 2017 07:49:59 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 292A6C05D8 for ; Fri, 1 Sep 2017 07:49:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.401 X-Spam-Level: X-Spam-Status: No, score=-0.401 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id K4itjJqyLNb3 for ; Fri, 1 Sep 2017 07:49:57 +0000 (UTC) Received: from mail-oi0-f43.google.com (mail-oi0-f43.google.com [209.85.218.43]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 886C45F6BF for ; Fri, 1 Sep 2017 07:49:57 +0000 (UTC) Received: by mail-oi0-f43.google.com with SMTP id r203so15045039oih.0 for ; Fri, 01 Sep 2017 00:49:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=2OiFFsMW0Pk0Rr2CjLV+WUmhUwpws39+BnOOuGp7Wb8=; b=os4imAheD8pyvg4FV3trsQRX3mhaHtPKA6jvWAO6ATjzXA/rmH8yZOqOZkmUPL/pmJ MxTHPGyHd1LEmnzwpwTwBSo6MEwkj7STVMtHJBGcmvrE1JbxaO/DMaacvfM1+xkGgaWo QTsbMgeG0fGoy5h/qNnk/efvU4s8+ej6tt9vAW/a5gY1dSHjS9XjWb92lhbTswAhT6ZP obWxP3j/HwBgUbqMRwGJD0PrVbOYXgPmEv9JRUGrEvsFHfgPomhiwtBqz/8q3wgl2yl3 TZaMSjJVf8i3u3rD1m2RcrivVPmVLOfFVvMGglG/4Dfkb5moRQMPPYz0c0bzpPwzchVr mB+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=2OiFFsMW0Pk0Rr2CjLV+WUmhUwpws39+BnOOuGp7Wb8=; b=L0t7AVpmjUOhBfJN8gUGL1AD2clgyPGXSS+0H59NL7ms3nUyEgAY6ANr4aCrbGe1YE +x/6RDjko14PkZ4X9EQL12LpmZvp/dC6l/KFA3svt97FYtmP40uUPwlgd+D9v8uZtWZ/ IpKCqroyfMuXbJ8pL2S3tqUVkdriE8StVYWvPvpZFUAxChKKp6wRGL0r/vImtGbtUR1C TgDnWh8ZdxNvnPdkVkOp6B2TK9ASMEdLM+G0BG+Er4TbOpCjD4qvKUOxO/3MjBaWG5iY kmS8tPtZr2lk8tv99VnGYzUo+UU7+Bq2JugfLO4AcxHvXuGA9F+GBSJ7vIm6Jwz7uCav uibQ== X-Gm-Message-State: AHPjjUgRe8EOVHQ0ZhR7Bv1hnmaWnJVDUAirt9vnwTG1+7sam8iJFvio ics++DIP1BDFYf7oK7tfpESbCeVKIC9r7Iw= X-Google-Smtp-Source: ADKCNb6w6SxEi26+Q9cflKQqc6nfKtqNvY8rhUeysndiXjJ0B4POfPj+nWWZ3dwJ5QVyM70lcf7cm0lVHZFchDO1IYw= X-Received: by 10.202.45.71 with SMTP id t68mr932664oit.293.1504252190866; Fri, 01 Sep 2017 00:49:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.46.72 with HTTP; Fri, 1 Sep 2017 00:49:30 -0700 (PDT) In-Reply-To: <7210840B-6C97-4DA0-9FEA-79AF3D18F165@gmail.com> References: <2DF54BB2-2A0C-4F56-8342-8BEF3AFEB935@gmail.com> <7210840B-6C97-4DA0-9FEA-79AF3D18F165@gmail.com> From: Nicolas Guyomar Date: Fri, 1 Sep 2017 09:49:30 +0200 Message-ID: Subject: Re: old big tombstone data file occupy much disk space To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary="001a1137d20c7d45b005581c01cb" archived-at: Fri, 01 Sep 2017 07:50:02 -0000 --001a1137d20c7d45b005581c01cb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Last time I used forceUserDefinedCompaction, I got myself a headache because I was trying to use a full path like you're doing, but in fact it just need the sstable as parameter Can you just try : echo "run -b org.apache.cassandra.db:type=3DCompactionManager forceUserDefinedCompaction mc-100963-big-Data.db" | java -jar /opt/cassandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar -l localhost:7199 On 1 September 2017 at 08:29, qf zhou wrote: > > dataPath=3D/hdd3/cassandra/data/gps/gpsfullwithstate- > 073e51a0cdb811e68dce511be6a305f6/mc-100963-big-Data.db > echo "run -b org.apache.cassandra.db:type=3DCompactionManager > forceUserDefinedCompaction $dataPath" | java -jar /opt/cassandra/tools/jm= x/jmxterm-1.0-alpha-4-uber.jar > -l localhost:7199 > > In the above, I am using a jmx method. But it seems that the file size > doesn=E2=80=99t change. My command is wrong ? > > > =E5=9C=A8 2017=E5=B9=B49=E6=9C=881=E6=97=A5=EF=BC=8C=E4=B8=8B=E5=8D=882= :17=EF=BC=8CJeff Jirsa =E5=86=99=E9=81=93=EF=BC=9A > > > > User defined compaction to do a single sstable compaction on just that > sstable > > > > It's a nodetool command in very recent versions, or a jmx method in > older versions > > > > > > -- > > Jeff Jirsa > > > > > >> On Aug 31, 2017, at 11:04 PM, qf zhou wrote: > >> > >> I am using a cluster with 3 nodes and the cassandra version is > 3.0.9. I have used it about 6 months. Now each node has about 1.5T data i= n > the disk. > >> I found some sstables file are over 300G. Using the sstablemetadata > command, I found it: Estimated droppable tombstones: 0.9622972799707109= . > >> It is obvious that too much tombstone data exists. > >> The default_time_to_live =3D 8640000(100 days) and gc_grace_seconds = =3D > 432000(5 days). Using nodetool compactionstats, I found the some > compaction processes exists. > >> So I really want to know how to clear tombstone data ? otherwise the > disk space will cost too much. > >> I really need some help, because some few people know cassandra in my > company. > >> Thank you very much! > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org > >> For additional commands, e-mail: user-help@cassandra.apache.org > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org > > For additional commands, e-mail: user-help@cassandra.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org > For additional commands, e-mail: user-help@cassandra.apache.org > > --001a1137d20c7d45b005581c01cb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Last time I used=C2=A0forceUserDefinedCompaction, I got myself a headache b= ecause I was trying to use a full path like you're doing, but in fact i= t just need the sstable as parameter

Can yo= u just try :=C2=A0

echo "run -b org.apache.cassandra.db:type=3DCompactionManager forceUser= DefinedCompaction=C2=A0mc-100963-bi= g-Data.db" | java -jar /opt/ca= ssandra/tools/jmx/jmxterm-1.0-alpha-4-uber.jar=C2=A0 =C2=A0-l localhost:7199



On 1 September 2017 at 08:29, qf zhou <zhouqf2013@gmail.com> wrote:

dataPath=3D/hdd3/cassandra/data/gps/gpsfullwithstate-073e51a0cdb8= 11e68dce511be6a305f6/mc-100963-big-Data.db
echo "run -b org.apache.cassandra.db:type=3DCompactionManager for= ceUserDefinedCompaction $dataPath" | java -jar /opt/cassandra/tools/jm= x/jmxterm-1.0-alpha-4-uber.jar=C2=A0 =C2=A0-l localhost:7199

In the above, I am using a jmx method. But it seems that the file size does= n=E2=80=99t change. My command is wrong ?

> =E5=9C=A8 2017=E5=B9=B49=E6=9C=881=E6=97=A5=EF=BC=8C=E4=B8=8B=E5=8D=88= 2:17=EF=BC=8CJeff Jirsa <jjirsa@gmai= l.com> =E5=86=99=E9=81=93=EF=BC=9A
>
> User defined compaction to do a single sstable compaction on just that= sstable
>
> It's a nodetool command in very recent versions, or a jmx method i= n older versions
>
>
> --
> Jeff Jirsa
>
>
>> On Aug 31, 2017, at 11:04 PM, qf zhou <zhouqf2013@gmail.com> wrote:
>>
>> I am using=C2=A0 a cluster with=C2=A0 3 nodes and=C2=A0 the cassan= dra version is 3.0.9. I have used it about 6 months. Now each node has abou= t 1.5T data in the disk.
>> I found some sstables file are over 300G. Using the=C2=A0 sstablem= etadata command,=C2=A0 I found it:=C2=A0 Estimated droppable tombstones: 0.= 9622972799707109.
>> It is obvious that too much tombstone data exists.
>> The default_time_to_live =3D 8640000(100 days) and=C2=A0 =C2=A0gc_= grace_seconds =3D 432000(5 days).=C2=A0 Using nodetool=C2=A0 compactionstat= s, I found the some compaction processes exists.
>> So I really=C2=A0 want to know how to clear tombstone data ?=C2=A0= otherwise the disk space will cost too much.
>> I really need some help, because some few people know cassandra in= my company.
>> Thank you very much!
>>
>>
>> ------------------------------------------------------------<= wbr>---------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
>
> ------------------------------------------------------------= ---------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>


-----------------------------------------------------------------= ----
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org


--001a1137d20c7d45b005581c01cb--