hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tianyin Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15029) A number of inconsistent defaults of configuration parameters
Date Wed, 23 Dec 2015 01:42:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15069013#comment-15069013
] 

Tianyin Xu commented on HBASE-15029:
------------------------------------

I guess so... I was really bugged by an issue caused by such inconsistency in my cluster (as
I usually only read the docs)... so I decide to write a script and then I find many configs
in docs are not the values I thought to be (so surprising)... >.<
I'll give a patch shortly (going out for dinner...). 

Thanks a lot, [~anoop.hbase]!



> A number of inconsistent defaults of configuration parameters
> -------------------------------------------------------------
>
>                 Key: HBASE-15029
>                 URL: https://issues.apache.org/jira/browse/HBASE-15029
>             Project: HBase
>          Issue Type: Bug
>          Components: defaults, documentation, hbase
>    Affects Versions: 1.1.2
>            Reporter: Tianyin Xu
>
> In HBase, a list of default configuration values are inconsistent with what is described
in the docs ({{hbase-default.xml}}) and the online docs (http://hbase.apache.org/book.html#config.files).

> This is really confusing to users like me, as we usually only rely on the docs. 
> *1. {{hbase.lease.recovery.dfs.timeout}}*
> In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],
the value is {{64000}}, while it is {{61000}} in the code,
> {code:title=FSHDFSUtils.java (the only usage)|borderStyle=solid}
>     // This should be set to how long it'll take for us to timeout against primary datanode
if it
>     // is dead.  We set it to 61 seconds, 1 second than the default READ_TIMEOUT in HDFS,
the
>     // default value for DFS_CLIENT_SOCKET_TIMEOUT_KEY. If recovery is still failing
after this
>     // timeout, then further recovery will take liner backoff with this base, to avoid
endless
>     // preemptions when this value is not properly configured.
>     long subsequentPauseBase = conf.getLong("hbase.lease.recovery.dfs.timeout", 61 *
1000);
> {code}
> The comments confirm that the value should be {{61000}} rather than {{64000}}. 
> \\
> \\
> *2. {{hbase.client.retries.number}}*
> In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],
the value is {{10}}, while it is {{31}} in the code,
> {code:title=HConstants.java|borderStyle=solid}
>   /**
>    * Default value of {@link #HBASE_CLIENT_RETRIES_NUMBER}.
>    */
>   public static final int DEFAULT_HBASE_CLIENT_RETRIES_NUMBER = 31;
> {code}
> All the places using {{hbase.client.retries.number}} take {{DEFAULT_HBASE_CLIENT_RETRIES_NUMBER}}
as the default value.
> \\
> \\
> *3. {{zookeeper.session.timeout}}*
> In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],
the value is {{90000}}, while it is {{180000}} in the code,
> {code:title=HConstants.java|borderStyle=solid}
>   /** Configuration key for ZooKeeper session timeout */
>   public static final String ZK_SESSION_TIMEOUT = "zookeeper.session.timeout";
>   /** Default value for ZooKeeper session timeout */
>   public static final int DEFAULT_ZK_SESSION_TIMEOUT = 180 * 1000;
> {code}
> All the places using {{zookeeper.session.timeout}} take {{DEFAULT_ZK_SESSION_TIMEOUT}}.
> \\
> \\
> *4. {{hbase.hstore.blockingStoreFiles}}*
> In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],
the value is {{10}}, while it is {{7}} in the code,
> {code:title=HStore.java|borderStyle=solid}
>   public static final String BLOCKING_STOREFILES_KEY = "hbase.hstore.blockingStoreFiles";
>   public static final int DEFAULT_BLOCKING_STOREFILE_COUNT = 7;
> {code}
> Same as #3 and #4.
> \\
> \\
> *5. {{hbase.http.max.threads}}*
> In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],
the value is {{10}}, while {{10}} is never used in the code.
> The code logic is a little bit complex:
> {code:title=HttpServer.java  (the only usage)|borderStyle=solid}
>   static final String HTTP_MAX_THREADS = "hbase.http.max.threads";
>   ...
>     int maxThreads = conf.getInt(HTTP_MAX_THREADS, -1);
>     // If HTTP_MAX_THREADS is not configured, QueueThreadPool() will use the
>     // default value (currently 250).
>     QueuedThreadPool threadPool = maxThreads == -1 ? new QueuedThreadPool()
>         : new QueuedThreadPool(maxThreads);
> {code}
> Basically, if the user does not set the value, {{maxThreads}} would be {{-1}} which is
equavalent to the default value of {{_maxThreads}} in the {{QueuedThreadPool}} class (it is
{{250}} right now).
> \\
> \\
> *6. {{hbase.ipc.server.callqueue.handler.factor}}*
> In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],
the value is {{0.1}}, while it is {{0}} in the code,
> {code:title=SimpleRpcScheduler.java|borderStyle=solid}
>   public static final String CALL_QUEUE_HANDLER_FACTOR_CONF_KEY =
>       "hbase.ipc.server.callqueue.handler.factor";
>   ...
>     float callQueuesHandlersFactor = conf.getFloat(CALL_QUEUE_HANDLER_FACTOR_CONF_KEY,
0);
>     int numCallQueues = Math.max(1, (int)Math.round(handlerCount * callQueuesHandlersFactor));
> {code}
> \\
> *7. {{hbase.regionserver.logroll.errors.tolerated}}*
> In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],,
the value is {{2}}, while it is {{0}} in the code (i.e., the default is not tolerating any
error...),
> {code:title=FSHLog.java  (the only usage)|borderStyle=solid}
>     this.closeErrorsTolerated = conf.getInt("hbase.regionserver.logroll.errors.tolerated",
0);
> {code}
> This is really confusing as we suppose to have 2 errors tolerated...
> \\
> \\
> *8. {{hbase.snapshot.enabled}}*
> I'm not really confident for this one. In hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],,
the value is {{true}}, while it is {{false}} in the code
> {code:title=SnapshotManager.java  (the only usage)|borderStyle=solid}
>   public static final String HBASE_SNAPSHOT_ENABLED = "hbase.snapshot.enabled";
>    ...
>     boolean snapshotEnabled = conf.getBoolean(HBASE_SNAPSHOT_ENABLED, false);
> {code}
> \\
> *9. {{hbase.online.schema.update.enable}}*
> Again, in both hbase-default.xml and [online docs|http://hbase.apache.org/book.html#config.files],,
the value is {{true}}, while it is {{false}} in both {{TableEventHandler}} and {{MasterDDLOperationHelper}}:
> {code:title=TableEventHandler.java|borderStyle=solid}
>   private boolean isOnlineSchemaChangeAllowed() {
>     return this.server.getConfiguration().getBoolean(
>       "hbase.online.schema.update.enable", false);
>   }
> {code}
> {code:title=MasterDDLOperationHelper.java|borderStyle=solid}
>   public static boolean isOnlineSchemaChangeAllowed(final MasterProcedureEnv env) {
>     return env.getMasterServices().getConfiguration()
>         .getBoolean("hbase.online.schema.update.enable", false);
>   }
> {code}
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message