Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <MWHPR14MB1293D533A1613E9F73A31DB6C13C0@MWHPR14MB1293.namprd14.prod.outlook.com>
References: <MWHPR14MB1293D533A1613E9F73A31DB6C13C0@MWHPR14MB1293.namprd14.prod.outlook.com>
From: Vladimir Rodionov <vladrodionov@gmail.com>
Date: Wed, 22 Mar 2017 12:36:05 -0700
Message-ID: <CAAg3a2o7wxfgNirV7ykA392UOYN9vwD1Nz-TRkC=K50NvuQO=w@mail.gmail.com>
Subject: Re: Need guidance on Custom Compaction Policy
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a1147a9ba12cbde054b56dfeb
archived-at: Wed, 22 Mar 2017 19:36:26 -0000

--001a1147a9ba12cbde054b56dfeb
Content-Type: text/plain; charset=UTF-8

Older files will be purged by default HBase compactor if all data inside
expired (you have TTL for data?)

As for custom compaction policy you can refer to FIFOCompactionPolicy class
to get the idea how custom compaction works.

-Vlad


On Wed, Mar 22, 2017 at 12:29 PM, jeff saremi <jeffsaremi@hotmail.com>
wrote:

> I mentioned some of this in another thread. We have a readonly database
> which get bulk loaded using HFiles.
> We want to keep only two versions/generations of data. Since the size of
> data is massive we need to delete the older generation.
>
> Since we write one single HBase for each region for each CF for each
> generation, could we just yank the older files using a separate standalone
> process? (sounds a little scary)
>
> If not, could we write a custom compactor? what's involved (some pointers
> please)? thanks
>
> Jeff
>

--001a1147a9ba12cbde054b56dfeb--