hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7236) add per-table/per-cf configuration via metadata
Date Thu, 13 Dec 2012 19:50:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531396#comment-13531396

Sergey Shelukhin commented on HBASE-7236:

bq. Attributes as in TTL, MIN_VERSIONS, which are part of the hbase data model
Actually, these are not part of data model, this is config.
For many existing metadata settings, it makes perfect sense to have a cluster-wide value (e.g.
TTL, versions, compression parameters, etc.). 
IMHO metadata is stuff like IS_ROOT, IS_READONLY, etc.
So we have 3 types of values, excluding schema:
1) Metadata specific to column/table, e.g. IS_ROOT.
2) Config:
  a) Some of config is xml-file only and has names like hbase.foo.bar.baz.
  b) Some config is descriptor-only and has names like MIN_VERSIONS.
  c) Some config is both xml and column-specific, and has names like hbase.region.max.filesize
(or something) *and* MAX_FILESIZE.
3) User-specific parameters, set via CONFIG in shell, and used for mysterious user-specific
purposes (right now, CF user parameters can also override xml config implicitly, which I am
not removing with this patch to keep backward compat).

I don't see a reason not to separate all of this. 
This patch cleans up 2c and changes it to a general mechanism to do such things, instead of
case-by-case basis which as shown above leads to more code and potential bugs.

Ideally, I'd say we should:
* remove 2b in a sense that all the config values should be specifiable in "hbase.foo.bar.baz"
form, and both in XML file (e.g. for example "hbase.cf.min.versions"), and table/cf (where
it makes sense; e.g. we can whitelist overridable keys).
* Separate (1) from (3) to prevent potential of conflicts if we add new reserved keywords
that user has set on his descriptors for their own purposes, but that's a separate issue.

bq. Is putting arbitrary byte[]'s into the table/column descriptors a valid use case for the
Yes, there were JIRAs about improving it so I assume it's used :)
> add per-table/per-cf configuration via metadata
> -----------------------------------------------
>                 Key: HBASE-7236
>                 URL: https://issues.apache.org/jira/browse/HBASE-7236
>             Project: HBase
>          Issue Type: New Feature
>          Components: Compaction
>    Affects Versions: 0.96.0
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HBASE-7236-PROTOTYPE.patch, HBASE-7236-PROTOTYPE.patch, HBASE-7236-PROTOTYPE-v1.patch,
HBASE-7236-v0.patch, HBASE-7236-v1.patch, HBASE-7236-v2.patch, HBASE-7236-v3.patch
> Regardless of the compaction policy, it makes sense to have separate configuration for
compactions for different tables and column families, as their access patterns and workloads
can be different. In particular, for tiered compactions that are being ported from 0.89-fb
branch it is necessary to have, to use it properly.
> We might want to add support for compaction configuration via metadata on table/cf.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message