9.7.4. Region Splits

Splits run unaided on the RegionServer; i.e. the Master does not participate. The RegionServer splits a region, offlines the split region and then adds the daughter regions to META, opens daughters on the parent's hosting RegionServer and then reports the split to the Master. See Section 2.5.2.7, “Managed Splitting” for how to manually manage - splits (and for why you might do this)

9.7.4.1. Custom Split Policies

The default split policy can be overwritten using a custom RegionSplitPolicy (HBase 0.94+). + splits (and for why you might do this)

9.7.4.1. Custom Split Policies

The default split policy can be overwritten using a custom RegionSplitPolicy (HBase 0.94+). Typically a custom split policy should extend HBase's default split policy: ConstantSizeRegionSplitPolicy.

The policy can set globally through the HBaseConfiguration used or on a per table basis:

@@ -74,7 +74,7 @@ myHtd.setValue(HTableDescriptor.SPLIT_PO
        When asked to flush, current memstore is moved to snapshot and is cleared.
        HBase continues to serve edits out of new memstore and backing snapshot until flusher reports in that the
        flush succeeded. At this point the snapshot is let go.

9.7.5.2. StoreFile (HFile)

StoreFiles are where your data lives. -

9.7.5.2.1. HFile Format

The hfile file format is based on +

9.7.5.2.1. HFile Format

The hfile file format is based on the SSTable file described in the BigTable [2006] paper and on Hadoop's tfile (The unit test suite and the compression harness were taken directly from tfile). Modified: hbase/hbase.apache.org/trunk/book/regionserver.arch.html URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/regionserver.arch.html?rev=1462679&r1=1462678&r2=1462679&view=diff ============================================================================== --- hbase/hbase.apache.org/trunk/book/regionserver.arch.html (original) +++ hbase/hbase.apache.org/trunk/book/regionserver.arch.html Sat Mar 30 00:19:55 2013 @@ -61,22 +61,22 @@ For more general information about the concept of write ahead logs, see the Wikipedia Write-Ahead Log article.

9.6.5.2. WAL Flushing

TODO (describe). -

9.6.5.3. WAL Splitting

9.6.5.3.1. How edits are recovered from a crashed RegionServer

When a RegionServer crashes, it will lose its ephemeral lease in - ZooKeeper...TODO

9.6.5.3.2. `hbase.hlog.split.skip.errors`

When set to true, any error +

9.6.5.3. WAL Splitting

9.6.5.3.1. How edits are recovered from a crashed RegionServer

When a RegionServer crashes, it will lose its ephemeral lease in + ZooKeeper...TODO

9.6.5.3.2. `hbase.hlog.split.skip.errors`

When set to true, any error encountered splitting will be logged, the problematic WAL will be moved into the .corrupt directory under the hbase rootdir, and processing will continue. If set to false, the default, the exception will be propagated and the - split logged as failed.^[22]

9.6.5.3.3. How EOFExceptions are treated when splitting a crashed + split logged as failed.^[22]

9.6.5.3.3. How EOFExceptions are treated when splitting a crashed RegionServers' WALs

If we get an EOF while splitting logs, we proceed with the split even when hbase.hlog.split.skip.errors == false. An EOF while reading the last log in the set of files to split is near-guaranteed since the RegionServer likely crashed mid-write of a record. But we'll continue even if we got an - EOF reading other than the last file in the set.^[23]

^[22]See HBASE-2958 + EOF reading other than the last file in the set.^[23]

^[22]See HBASE-2958 When hbase.hlog.split.skip.errors is set to false, we fail the split but thats it. We need to do more than just fail split - if this flag is set.

^[23]For background, see HBASE-2643 + if this flag is set.

^[23] For background, see HBASE-2643 Figure how to deal with eof splitting logs

comments powered by Disqus

Prev		Next
11.12. Case Studies	Home	12.2. Logs

\ No newline at end of file +comments powered by Disqus

Prev		Next
11.13. Case Studies	Home	12.2. Logs

\ No newline at end of file Modified: hbase/hbase.apache.org/trunk/book/trouble.resources.html URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/trouble.resources.html?rev=1462679&r1=1462678&r2=1462679&view=diff ============================================================================== --- hbase/hbase.apache.org/trunk/book/trouble.resources.html (original) +++ hbase/hbase.apache.org/trunk/book/trouble.resources.html Sat Mar 30 00:19:55 2013 @@ -8,12 +8,12 @@ is generally used for questions on released versions of Apache HBase. Before going to the mailing list, make sure your question has not already been answered by searching the mailing list archives first. Use Section 12.3.1, “search-hadoop.com”. - Take some time crafting your question^[28]; a quality question that includes all context and + Take some time crafting your question^[28]; a quality question that includes all context and exhibits evidence the author has tried to find answers in the manual and out on lists is more likely to get a prompt response.

12.3.3. IRC

#hbase on irc.freenode.net

12.3.4. JIRA

JIRA is also really helpful when looking for Hadoop/HBase-specific issues. -

^[28]See Getting Answers

9.7.4. Region Splits

9.7.4. Region Splits

9.7.4.1. Custom Split Policies

9.7.4.1. Custom Split Policies

9.7.5.2. StoreFile (HFile)

9.7.5.2.1. HFile Format

9.7.5.2.1. HFile Format

9.6.5.2. WAL Flushing

9.6.5.3. WAL Splitting

9.6.5.3.1. How edits are recovered from a crashed RegionServer

9.6.5.3.2. hbase.hlog.split.skip.errors

9.6.5.3. WAL Splitting

9.6.5.3.1. How edits are recovered from a crashed RegionServer

9.6.5.3.2. hbase.hlog.split.skip.errors

9.6.5.3.3. How EOFExceptions are treated when splitting a crashed + split logged as failed.[22]

9.6.5.3.3. How EOFExceptions are treated when splitting a crashed RegionServers' WALs

12.3.3. IRC

12.3.4. JIRA

9.6.5.3.2. `hbase.hlog.split.skip.errors`

9.6.5.3.2. `hbase.hlog.split.skip.errors`

9.6.5.3.3. How EOFExceptions are treated when splitting a crashed + split logged as failed.^[22]