hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingcheng Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13670) [HBase MOB] ExpiredMobFileCleaner tool is not deleting the expired mob data.
Date Thu, 14 May 2015 03:08:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14543121#comment-14543121
] 

Jingcheng Du commented on HBASE-13670:
--------------------------------------

Do you have cases of TTL is smaller than one day, or just for the test convenience?
We make the date (not timestamp) in file name on purpose.
# In almost all the cases, the TTL is much larger than one day.
# Make sure the file name is readable, at the same time make sure the file name is not long.
If we have timestamp in the file name, we have to convert it to a hex string to make it readable,
it will take at least 26 bytes whereas now we only have 8 bytes.

The 2nd point makes the timestamp in the name be a problem. I don't think it is possible to
have timestamp in the file name. Thanks.

> [HBase MOB] ExpiredMobFileCleaner tool is not deleting the expired mob data.
> ----------------------------------------------------------------------------
>
>                 Key: HBASE-13670
>                 URL: https://issues.apache.org/jira/browse/HBASE-13670
>             Project: HBase
>          Issue Type: Bug
>          Components: mob
>    Affects Versions: hbase-11339
>            Reporter: Y. SREENIVASULU REDDY
>             Fix For: hbase-11339
>
>
> ExpiredMobFileCleaner tool is not deleting the expired mob data.
> steps to reproduce:
> ===============
> 1.Create the table with one column family as mob and set the TTL for mob columnfamily
very less.
> {code}
> hbase(main):020:0> describe 'mobtab'
> Table mobtab is ENABLED
> mobtab
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'mobcf',  IS_MOB => 'true',MOB_THRESHOLD => '102400', VERSIONS =>
'1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => '60 SECONDS
(1 MINUTE)', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BL
> OOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE
=> 'true', BLOCKSIZE => '65536'}
> {NAME => 'norcf', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false',
KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE',
TTL => 'FOREVER', MIN_VERSIONS => '0', BL
> OCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
> 2 row(s) in 0.0650 seconds
> {code}
> 2. then insert the mob data into the table(mobcf), and normal data into the another columnFamily(norcf).
> 3. flush the table.
> 4. scan the table before TTL expire. (able to fetch the data)
> 5. scan the table after TTL got expired, as a result mob data should not display, and
mob file should exist in hdfs.
> 5. run ExpiredMobFileCleaner tool manually to clean the expired mob data for TTL expired
data.
> {code}
> ./hbase org.apache.hadoop.hbase.mob.ExpiredMobFileCleaner mobtab mobcf
> {code}
> {code}
> client log_message:
> 2015-05-09 18:03:37,731 INFO  [main] mob.ExpiredMobFileCleaner: Cleaning the expired
MOB files of mobcf in mobtab
> 2015-05-09 18:03:37,734 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
> 2015-05-09 18:03:37,738 INFO  [main] mob.MobUtils: MOB HFiles older than 8 May 2015 18:30:00
GMT will be deleted!
> 2015-05-09 18:03:37,971 DEBUG [main] mob.MobUtils: Checking file d41d8cd98f00b204e9800998ecf8427e20150509c9108e1a9252418abbfd54323922c518
> 2015-05-09 18:03:37,971 INFO  [main] mob.MobUtils: 0 expired mob files are deleted
> 2015-05-09 18:03:37,971 INFO  [main] client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
> {code}
> *problem:*
> If we run ExpiredMobFileCleaner tool manually, it is not deleting the expired mob data.
For deletion it is considering default time period "hbase.master.mob.ttl.cleaner.period".
> With this Time period "hbase.master.mob.ttl.cleaner.period" only ExpiredMobFileCleanerChore
should consider.
> {code}
> conf:
> <property>
> <name>hbase.master.mob.ttl.cleaner.period</name>
> <value>86400000</value>
> <source>hbase-default.xml</source>
> </property>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message