Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of jgray@fb.com designates
 66.220.144.150 as permitted sender)
From: Jonathan Gray <jgray@fb.com>
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Subject: RE: Online modify table schema (e.g. AddColumnFamily,
 DeleteColumnFamily, etc) and other features
Thread-Topic: Online modify table schema (e.g. AddColumnFamily,
 DeleteColumnFamily, etc) and other features
Thread-Index: AQHLuMMex7PYjczL10GAMHBNt55b/ZPaG5og
Date: Thu, 20 Jan 2011 18:10:11 +0000
Message-ID: 
 <5A76F6CE309AD049AAF9A039A39242820F665A08@sc-mbx04.TheFacebook.com>
References: <AANLkTin_k=d5HFiW_+gENk_AewgyBSSPt6UvZWdk6-HB@mail.gmail.com>
In-Reply-To: <AANLkTin_k=d5HFiW_+gENk_AewgyBSSPt6UvZWdk6-HB@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

> I have following questions about some features which are needed in my
> applications.
>=20
> 1. The current HBase release(0.89, 0.90 ~) does not support to modify tab=
le
> schema online.
>      If I want add or delete a ColumnFamily, I must disable/offline the t=
able
> firstly.
>      Is there a plan to support online modification?
>      I think after the table schema info is moved into ZooKeeper from
> RegionServer, this feature is possible to be supported.

This is in the plans and exactly as you describe.  We are planning to move =
table schema information into ZooKeeper and then support online schema modi=
fications.

I'm unsure if this is currently targeted at 0.92 or not, but I think the pl=
an is to be aggressive on releasing 0.92 soon.  This would represent a reas=
onably large change so might get punted into 0.94 but hopefully no later th=
an that.

Of course, this is an open source project, so if there are developers who n=
eed this then all contributions are welcome.  There are some open JIRAs rel=
ated to this and I think a good bit of code already written (but based on t=
he old master).

See HBASE-1730 for more info...

https://issues.apache.org/jira/browse/HBASE-1730


> 2. Different memstore/memtable-size and region-size for different table.
>     This feature is useful for different tables for different application=
s.
>      Is there a plan to support it?

I agree that these settings can be useful on a per-table setting (and where=
 applicable, a per-family setting).

I'm not aware of anyone currently looking into these specific settings but =
again, just need a developer to go after it and implement it :)


>=20
> 3. Periodic flush.
>     Current HBase only flush memstore to HFile by memory size thresholds.
>     If a Region-Store is quiet for a long time (such as 2 hours) for a wr=
ite-dense
> application, the periodic flush can free more memory.
>     Is there a plan to support it?

I'm unsure what you're describing.  If there is a write-dense application, =
then MemStores will be flushed as they get filled or as the total MemStore =
usage across all regions goes above the max allowed capacity given availabl=
e heap.  If write-heavy workload, then MemStores should constantly be fille=
d and should constantly be flushed.

What would the benefit be of flushing if no writes for a certain period?  T=
his is to free aggregate MemStore space for frequently written to regions w=
hen there are less frequently written-to regions that are idle but taking u=
p MemStore space?

JG