hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14436) HTableDescriptor#addCoprocessor will always make RegionCoprocessorHost create new Configuration
Date Wed, 07 Oct 2015 02:38:26 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946154#comment-14946154
] 

Hudson commented on HBASE-14436:
--------------------------------

SUCCESS: Integrated in HBase-0.98 #1145 (See [https://builds.apache.org/job/HBase-0.98/1145/])
HBASE-14436 HTableDescriptor#addCoprocessor will always make (stack: rev 6c0f501ece9b7c31f6ce64c85289e55b4f7cc875)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java


> HTableDescriptor#addCoprocessor will always make RegionCoprocessorHost create new Configuration
> -----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14436
>                 URL: https://issues.apache.org/jira/browse/HBASE-14436
>             Project: HBase
>          Issue Type: Improvement
>          Components: Coprocessors
>    Affects Versions: 1.2.1
>            Reporter: Jianwei Cui
>            Assignee: stack
>            Priority: Minor
>             Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
>         Attachments: HBASE-14436-trunk-v1.patch, HBASE-14436-trunk-v2.patch
>
>
> HTableDescriptor#addCoprocessor will set the coprocessor value as following format:
> {code}
>  public HTableDescriptor addCoprocessor(String className, Path jarFilePath,
>                              int priority, final Map<String, String> kvs)
>   throws IOException {
>   ...
>   String value = ((jarFilePath == null)? "" : jarFilePath.toString()) +
>         "|" + className + "|" + Integer.toString(priority) + "|" +
>         kvString.toString();
>   ...
> }
> {code}
> If the 'jarFilePath' is null,  the 'value' will always has the format '|className|priority|'
 even if 'kvs' is null, which means no extra arguments for the coprocessor. Then, in the server
side, RegionCoprocessorHost#getTableCoprocessorAttrsFromSchema will load the table coprocessors
as:
> {code}
>   static List<TableCoprocessorAttribute> getTableCoprocessorAttrsFromSchema(Configuration
conf,
>       HTableDescriptor htd) {
>     ...
>             try {
>               cfgSpec = matcher.group(4); // => cfgSpec will be '|' for the format
'|className|priority|'
>             } catch (IndexOutOfBoundsException ex) {
>               // ignore
>             }
>             Configuration ourConf;
>             if (cfgSpec != null) {  // => cfgSpec will be '|' for the format '|className|priority|'
>               ourConf = new Configuration(false);
>               HBaseConfiguration.merge(ourConf, conf);
>             }
>     ...
> }
> {code}
> The 'cfgSpec' will be '|' for the coprocessor formatted as '|className|priority|', so
that always create a new Configuration.
> In our production, there are a lot of tables having table-level coprocessors, so that
the region server will create new Configurations for each region of the table, this will consume
a certain number of memory when we have many such regions.
> To fix the problem, we can make the HTableDescriptor not append the '|' if no extra arguments
for the coprocessor, or check the 'cfgSpec' more strictly in server side which could avoid
creating new Configurations for existed such regions after the regions reopened. Discussions
and suggestions are welcomed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message