hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "steven zhuang (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-5505) if max file size set to 256M, Per table hbase.hregion.max.filesize setting(at htable creation) overwritten by value set in hbase-site.conf(which is not 256MB!)
Date Fri, 02 Mar 2012 02:46:59 GMT

     [ https://issues.apache.org/jira/browse/HBASE-5505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

steven zhuang updated HBASE-5505:
---------------------------------

    Description: 
seems to me that when set to DEFAULT_MAX_FILE_SIZE, the max region file size set at Htable
creation time may later be overwritten by value set in hbase-site.conf, i.e. no "Per table"
hbase.hregion.max.filesize at all.

in this case, on Web UI in the table description, max file is still 256M, but we can find
region file much bigger on HDFS.

In 0.92.0, o.a.h.hbase.regionserver.ConstantSizeRegionSplitPolicy, around line 33:

    long maxFileSize = region.getTableDesc().getMaxFileSize();

    // By default we split region if a file > HConstants.DEFAULT_MAX_FILE_SIZE.
    if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {         //zx: here setting is over
written by configuaration from hbase-site.conf
      maxFileSize = getConf().getLong("hbase.hregion.max.filesize",
        HConstants.DEFAULT_MAX_FILE_SIZE);
    }

In 0.90.4(cdh3u3) or earlier version, this is found in o.a.h.hbase.regionserver.Store, around
line 188:

    long maxFileSize = info.getTableDesc().getMaxFileSize();
    if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {
      maxFileSize = conf.getLong("hbase.hregion.max.filesize",
        HConstants.DEFAULT_MAX_FILE_SIZE);
    }

  was:
seems to me that when set as the DEFAULT_MAX_FILE_SIZE, the max region file size set at Htable
creation time may later be overwritten by value set in hbase-site.conf, i.e. no "Per table"
hbase.hregion.max.filesize at all.

in this case, on Web UI in the table description, max file is still 256M, but we can find
region file much bigger on HDFS.

In 0.92.0, o.a.h.hbase.regionserver.ConstantSizeRegionSplitPolicy, around line 33:

    long maxFileSize = region.getTableDesc().getMaxFileSize();

    // By default we split region if a file > HConstants.DEFAULT_MAX_FILE_SIZE.
    if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {         //zx: here setting is over
written by configuaration from hbase-site.conf
      maxFileSize = getConf().getLong("hbase.hregion.max.filesize",
        HConstants.DEFAULT_MAX_FILE_SIZE);
    }

In 0.90.4(cdh3u3) or earlier version, this is found in o.a.h.hbase.regionserver.Store, around
line 188:

    long maxFileSize = info.getTableDesc().getMaxFileSize();
    if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {
      maxFileSize = conf.getLong("hbase.hregion.max.filesize",
        HConstants.DEFAULT_MAX_FILE_SIZE);
    }

        Summary: if max file size set to 256M, Per table hbase.hregion.max.filesize setting(at
htable creation) overwritten by value set in hbase-site.conf(which is not 256MB!)  (was: if
max file size set to 256M, Per table hbase.hregion.max.filesize setting(at htable creation)
overwritten by value set in hbase-site.conf(might not be 256MB!))
    
> if max file size set to 256M, Per table hbase.hregion.max.filesize setting(at htable
creation) overwritten by value set in hbase-site.conf(which is not 256MB!)
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-5505
>                 URL: https://issues.apache.org/jira/browse/HBASE-5505
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.90.4, 0.92.0
>            Reporter: steven zhuang
>
> seems to me that when set to DEFAULT_MAX_FILE_SIZE, the max region file size set at Htable
creation time may later be overwritten by value set in hbase-site.conf, i.e. no "Per table"
hbase.hregion.max.filesize at all.
> in this case, on Web UI in the table description, max file is still 256M, but we can
find region file much bigger on HDFS.
> In 0.92.0, o.a.h.hbase.regionserver.ConstantSizeRegionSplitPolicy, around line 33:
>     long maxFileSize = region.getTableDesc().getMaxFileSize();
>     // By default we split region if a file > HConstants.DEFAULT_MAX_FILE_SIZE.
>     if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {         //zx: here setting
is over written by configuaration from hbase-site.conf
>       maxFileSize = getConf().getLong("hbase.hregion.max.filesize",
>         HConstants.DEFAULT_MAX_FILE_SIZE);
>     }
> In 0.90.4(cdh3u3) or earlier version, this is found in o.a.h.hbase.regionserver.Store,
around line 188:
>     long maxFileSize = info.getTableDesc().getMaxFileSize();
>     if (maxFileSize == HConstants.DEFAULT_MAX_FILE_SIZE) {
>       maxFileSize = conf.getLong("hbase.hregion.max.filesize",
>         HConstants.DEFAULT_MAX_FILE_SIZE);
>     }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message