hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Schubert Zhang <zson...@gmail.com>
Subject Re: Online modify table schema (e.g. AddColumnFamily, DeleteColumnFamily, etc) and other features
Date Fri, 21 Jan 2011 01:46:50 GMT
Thank you so much,  Jonathan Gray and Jean-Daniel Cryans,

1. We will try to do more research and try to do something for the
AddColumn/DeleteColumn issue.

2. It's my fault, I missed to check the HTableDescriptor metadata.
    It's great we can use MAX_FILESIZE and MEMSTORE_FLUSHSIZE to control
different table.

3. Yes, the total memstore heap threshold and the HLog.cleanOldLogs can
flush the old memstores.
    My scenario is:
    The columnfamilies/stores are defined by time-ranges (such days). So,
the columnfamilies/stores of yesterday will be quiet today.

Thank you again.
Schubert Zhang
On Fri, Jan 21, 2011 at 2:48 AM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> To complement what jgray answered...
>
> >
> > 2. Different memstore/memtable-size and region-size for different table.
> >    This feature is useful for different tables for different
> applications.
> >     Is there a plan to support it?
>
> Have you taken a look at the table properties? In the shell you can
> tune MAX_FILESIZE and MEMSTORE_FLUSHSIZE to do exactly what you
> described.
>
> >
> > 3. Periodic flush.
> >    Current HBase only flush memstore to HFile by memory size thresholds.
> >    If a Region-Store is quiet for a long time (such as 2 hours) for a
> > write-dense application, the periodic flush can free more memory.
> >    Is there a plan to support it?
>
> If a region isn't written to, and there's a write-heavy job going on,
> then as the HLogs get created it's easy to reach the max number of
> HLogs a RS can get (32 by default) and after that point the memstores
> for the oldest edits will be flushed. If it's really inserting a lot
> of data, I expect that it can takes less than an hour to force flush
> unused memstores. Check out HLog.cleanOldLogs and
> HLog.findMemstoresWithEditsEqualOrOlderThan
>
> It's not really periodic or easily configurable tho.
>
> J-D
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message