hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Ranganathan (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-3144) Freeze and thaw for regions and/or regionservers
Date Thu, 21 Oct 2010 21:57:14 GMT
Freeze and thaw for regions and/or regionservers

                 Key: HBASE-3144
                 URL: https://issues.apache.org/jira/browse/HBASE-3144
             Project: HBase
          Issue Type: New Feature
          Components: regionserver
            Reporter: Karthik Ranganathan

When frozen, a region would not flush new files or delete old files.

Was talking to Dhruba about the possibility of a fast intra-cluster copy, and he says it is
totally do-able. The idea is this: if there is a HBase table that is very large (multiple
terabytes) then distcp takes a long time. The major bottleneck is the cross rack network transfer
time. We can easily avoid this by going to each of the block replicas and replicating them
on the same machine from one disk to another. For our infrastructure setup, this could make
hours of copy time seconds.

If we are able to implement the above, then copying an entire HBase table will only take seconds.
Now we just have to ensure that the region files do not "change" during this period - so we
need to disable compactions and flushes. Hence the need for the freeze and thaw of regions
in a region server.

The way this would work is to flush a region, freeze it, snapshot it, then thaw it and let
normal operations proceed. The snapshot would proceed at disk read/write speed so it should
take only seconds per region.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message