hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8608) Add Configuration API for parsing time durations
Date Thu, 04 Oct 2012 08:49:08 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Douglas updated HADOOP-8608:
----------------------------------

    Attachment: 8608-1.patch

bq. I am kind of concerned about the "lose precision" from the conversion. That is why I propose
to give that control back to the caller to make the precision lose explicitly. If we go this
approach, at least I think we should document this precision lose. Comments?

The caller has exactly the same control in both models; the difference is whether the user
should be notified because they misinterpreted the knob. A warning when a conversion loses
some significant fraction of the value could be helpful, but is this common? When someone
makes a typo and sets a timeout as 10d instead of 10s, that seems more worthy of a warning,
but the caller knows better than the config if the value is in range.

It's a matter of taste, but I'd prefer to leave it with the caller. That OK?

bq. The parsing of value is a bit loose. For example, it cannot handle "10S" or "10 s". A
strict format can reduce errors but may be inflexible and the exception can be little harsh.
Or we need to document the expecting format is "10s" not "10 s" nor "10sec"

The format is documented and I prefer strict units, but feel free to make the matching fuzzier.

bq. Also the parsing part relies on the enum order implicitly. It works now. But it may bite
us later. A Pattern instead?

Bite how? The enum order is part of the spec. The parsing is dead-simple. This is a trivial
feature; a complex implementation is unlikely to justify itself...

bq. It would be better to add the value of the timeduration property into LOG.
bq. Can you please change test from "testTime" to "testTimeDuration" for consistency?

Sure.
                
> Add Configuration API for parsing time durations
> ------------------------------------------------
>
>                 Key: HADOOP-8608
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8608
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>    Affects Versions: 3.0.0
>            Reporter: Todd Lipcon
>         Attachments: 8608-0.patch, 8608-1.patch
>
>
> Hadoop has a lot of configurations which specify durations or intervals of time. Unfortunately
these different configurations have little consistency in units - eg some are in milliseconds,
some in seconds, and some in minutes. This makes it difficult for users to configure, since
they have to always refer back to docs to remember the unit for each property.
> The proposed solution is to add an API like {{Configuration.getTimeDuration}} which allows
the user to specify the units with a postfix. For example, "10ms", "10s", "10m", "10h", or
even "10d". For backwards-compatibility, if the user does not specify a unit, the API can
specify the default unit, and warn the user that they should specify an explicit unit instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message