hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "churro morales (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11360) SnapshotFileCache refresh logic based on modified directory time might be insufficient
Date Wed, 18 Jun 2014 22:56:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036598#comment-14036598
] 

churro morales commented on HBASE-11360:
----------------------------------------

So we could do this:

Create a configuration option called : hbase.snapshot.inProgress.maxTime which would allow
you to configure the maximum amount of time a snapshot should take

The refresh logic would then look to see 
Is there an inProgress token AND is current time - create timestamp for ".inProgress" <
hbase.snapshot.inProgress.maxTime
If so refresh the cache
Otherwise 
refresh the cache only if we have new snapshots in our snapshot directory (same logic as current)

I think that might be a safe way to go about things.  Anyways I'll think about it more tonight
and would love to hear your thoughts.




we could have an ".inProgress" token and check that as well as its timestamp to determine
whether or not we need a cache refresh.

on top of that you could create a configuration option say: hbase.snapshot.inProgress.maxTime
which would allow you to configure a timeout for that 

> SnapshotFileCache refresh logic based on modified directory time might be insufficient
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-11360
>                 URL: https://issues.apache.org/jira/browse/HBASE-11360
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.94.19
>            Reporter: churro morales
>
> Right now we decide whether to refresh the cache based on the lastModified timestamp
of all the snapshots and those "running" snapshots which is located in the /hbase/.hbase-snapshot/.tmp/<snapshot>
directory
> We ran a ExportSnapshot job which takes around 7 minutes between creating the directory
and copying all the files. 
> Thus the modified time for the 
> /hbase/.hbase-snapshot/.tmp directory was 7 minutes earlier than the modified time of
the
> /hbase/.hbase-snapshot/.tmp/<snapshot> directory
> Thus the cache refresh happens and doesn't pick up all the files but thinks its up to
date as the modified time of the .tmp directory never changes.
> This is a bug as when the export job starts the cache never contains the files for the
"running" snapshot and will fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message