hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9847) HDFS configuration without time unit name should accept friendly time units
Date Mon, 21 Mar 2016 21:27:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205191#comment-15205191

Chris Douglas commented on HDFS-9847:

Sorry, I overloaded "precision". By providing another parameter, it might provide semantics
like "give me the value of this parameter in this unit, unless it sheds more than (x% | x<unit>
| whatever) of the configured value". If the caller cares this much, they should ask for the
var with high precision and make the decision to throw in that context.

bq. Without the precision check Configuration silently changes values without the caller knowing
about it.

Isn't that the intent behind configuring with units? Throwing for loss of precision requires
the operator to know the default precision (no improvement over the current approach, which
embeds the unit in the config key), or go through at least one round of errors. To make this
less painful for users, the caller should only use {{getTimeDuration}} with high precision,
which gets us back to specifying everything but the string value in millis, internally. If
{{getTimeDuration}} throws, then why is it better?

I'd argue handling backwards compatibility is easier without throwing for loss of precision.
This warns for vars without units, which we've decided is sufficient for deprecated config
vars. New versions can increase precision changing only the timeunit param, and old client
configs will still work. If it wants to decrease precision, then the caller can warn/throw
for values that lose precision (whatever's appropriate for its context), and migrate to the
new unit in the next version.

> HDFS configuration without time unit name should accept friendly time units
> ---------------------------------------------------------------------------
>                 Key: HDFS-9847
>                 URL: https://issues.apache.org/jira/browse/HDFS-9847
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: 2.7.1
>            Reporter: Lin Yiqun
>            Assignee: Lin Yiqun
>         Attachments: HDFS-9847.001.patch, HDFS-9847.002.patch, HDFS-9847.003.patch, HDFS-9847.004.patch,
HDFS-9847.005.patch, HDFS-9847.006.patch, timeduration-w-y.patch
> In HDFS-9821, it talks about the issue of leting existing keys use friendly units e.g.
60s, 5m, 1d, 6w etc. But there are som configuration key names contain time unit name, like
{{dfs.blockreport.intervalMsec}}, so we can make some other configurations which without time
unit name to accept friendly time units. The time unit  {{seconds}} is frequently used in
hdfs. We can updating this configurations first.

This message was sent by Atlassian JIRA

View raw message