hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-1678) [hbase] On region split, master should designate which host should serve daughter splits
Date Tue, 14 Aug 2007 03:39:32 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jim Kellerman resolved HADOOP-1678.

    Resolution: Fixed

Tested changes on Fedora core 6 and Windows XP Pro (single and dual processor).
Committed changes.

Summary of changes:

Use new hbase.util.Writables: HBaseAdmin, HConnectionManager, HMaster,
HMerge, HRegion, HRegiondirReader, HRegionInfo, HRegionServer,
TestGet, TestScanner, TestScanner2, TestSplit

- don't throw IllegalStateException is region is offline and split

- added new meta column family members for recording split references

HLog, HStoreFile
- use new FileSystem.listPaths instead of deprecated one

HRegion, HMaster:
- move static functions from HRegion that were only used by the master
  (and required an ipc as opposed to just manipulating files) to HMaster
- Master now assigns children of split region instead of HRegionServer
  serving them automatically.

- use BatchUpdate for some communications with region server (more to do)
- hasReferences uses HRegionInterface instead of HTable
- changed some SortedMaps to Maps backed by HashMap since access to
  them was never in sort order.

- SplitOrCompactChecker keeps HTables for root and meta once
  instantiated on demand.
- use batch updates to update meta/root regions

- new static method returns byte array built from string of
  concatenation of row and column keys

- ClientScanner.close() now catches NotServingRegionException

- Remove some debug level log messages - they were just noise

Keying, Writables
- move some byte to X and X to byte methods from Keying to Writables

Unit tests:

TestScanner2, HRegion: move static functions only used by TestScanner2
from HRegion to TestScanner2

- now uses MiniHBaseCluster to start up cluster on top of
  MiniDFSCluster instead of starting up a MiniHBaseCluster on top of
  local file system

> [hbase] On region split, master should designate which host should serve daughter splits
> ----------------------------------------------------------------------------------------
>                 Key: HADOOP-1678
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1678
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/hbase
>    Affects Versions: 0.15.0
>            Reporter: stack
>            Assignee: Jim Kellerman
>             Fix For: 0.15.0
>         Attachments: patch.txt
> On region split, the daughter regions are deployed on the same host as served the split
parent.  This makes it so currently (unless the cluster is restarted), as a table grows, all
its regions remain on the one server.
> Instead, jurisdiction over who serves daughter splits should be passed to the master.
 If possible, before making a determination, the master should take into consideration current
cluster loadings and region distribution.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message