hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akmal Abbasov <akmal.abba...@icloud.com>
Subject Re: HBase snapshots and Compaction
Date Tue, 12 May 2015 21:06:31 GMT
Hi Ted,
Thank you for reply.
I am running with the default settings.

Sent from my iPhone

> On 12 May 2015, at 22:02, Ted Yu <yuzhihong@gmail.com> wrote:
> 
> Can you show us compaction related parameters you use ?
> 
> e.g. hbase.hregion.majorcompaction , hbase.hregion.majorcompaction.jitter ,
> etc
> 
> On Tue, May 12, 2015 at 9:52 AM, Akmal Abbasov <akmal.abbasov@icloud.com>
> wrote:
> 
>> HI,
>> I am using HBase 0.98.7.
>> I am using HBase snapshots to backup data. I create snapshot of tables
>> each our.
>> Each create snapshot process will cause the flush of the memstore, and
>> creation of hfiles.
>> When the number of hfiles will reach 3 the MINOR compaction process will
>> start for each CF.
>> Ok, I was expecting that the compaction will process only small hfiles,
>> and I won’t have problems with moving all data to archive folder
>> each time after compaction process ends.
>> But most of the times, the minor compaction is promoted to major(more than
>> 100 in 24 hours without loads).
>> As far as I know, the only possibility for this is that all hfiles are
>> eligible for compaction.
>> But when I tested the archive folder for a CF I see the strange situation
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 06:04
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/36dc06f4c34242daadc343d857a35734
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 06:04
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/7e8b993f97b84f4594542144f15b0a1e
>> -rw-r--r--   3 akmal supergroup      1.1 K 2015-05-10 06:04
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/b9afc64792ba4bf99a08f34033cc46ac
>> -rw-r--r--   3 akmal supergroup    638.4 K 2015-05-10 06:04
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/dff846ae4fc24d418289a95322b35d46
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 08:50
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/228eee22c32e458e8eb7f5d031f64b58
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 08:50
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/529257432308466f971e41db49ecffdf
>> -rw-r--r--   3 akmal supergroup    638.5 K 2015-05-10 08:50
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/839d3a6fc523435d8b44f63315fd11b8
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 08:50
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/8c245e8661b140439e719f69a535d57f
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 11:37
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/23497a31d3e721fe9b63c58fbe0224d5
>> -rw-r--r--   3 akmal supergroup    638.7 K 2015-05-10 11:37
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/8c9af0357d164221ad46b336cd660b30
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 11:37
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/8eb55b43c22d434954e2e0bfda656018
>> -rw-r--r--   3 akmal supergroup      1.0 K 2015-05-10 11:37
>> /hbase/archive/data/default/table1/0e8e3bf44a2ea5dfaa8a9c58d99b92e6/c/b8b6210d9e6d4ec2344238c6e9c17ddf
>> 
>> As I understood this files were copied to archive folder after compaction.
>> The part I didn’t understand is, why the file with 638 K was also selected
>> for compaction?
>> Any ideas?
>> Thank you.
>> 
>> Kind regards,
>> Akmal Abbasov
>> 
>> 
>> 

Mime
View raw message