hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
Date Tue, 01 Mar 2016 07:05:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173351#comment-15173351
] 

Anoop Sam John commented on HBASE-15321:
----------------------------------------

So what you observed is taking direct HDFS snapshot helps in time taken for this snapshot
op in setup.
As it is scan over region directly, it can use HBase kind of optimization like time range
check on files.   Make sense. Now I get why u want to use this way rather than MR over snapshot.

But on the patch, do we really need these changes?   We already have states like readOnly,
readEnabled, writeEnabled..  Adding one more read related makes things more confusing IMO.
{code}
HRegion r = HRegion.newHRegion(tableDir, wal, fs, conf, info, htd, null);
6361	    return r.openHRegion(null);
{code}
U can have this code in ur setup. Make the HRegionInfo to set it as a non primary replica
id.. Yes this HRegion instance will give out of date data just like non primary replicas.
  Setting to non primary make it to be readOnly.
bq.this.writestate.setReadOnly(ServerRegionReplicaUtil.isReadOnly(this));

Am I missing some other things?

> Ability to open a HRegion from hdfs snapshot.
> ---------------------------------------------
>
>                 Key: HBASE-15321
>                 URL: https://issues.apache.org/jira/browse/HBASE-15321
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 2.0.0
>            Reporter: churro morales
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15321-v1.patch, HBASE-15321-v2.patch, HBASE-15321-v3.patch,
HBASE-15321.patch
>
>
> Now that hdfs snapshots are here, we started to run our mapreduce jobs over hdfs snapshots.
 The thing is, hdfs snapshots are read-only point-in-time copies of the file system.  Thus
we had to modify the section of code that initialized the region internals in HRegion.   We
have to skip cleanup of certain directories if the HRegion is backed by a hdfs snapshot. 
I have a patch for trunk with some basic tests if folks are interested.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message