storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhiming Guo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (STORM-328) Config.java and Utils.get{Int,Long} are not in sync for floating point
Date Wed, 09 Jul 2014 01:23:04 GMT

    [ https://issues.apache.org/jira/browse/STORM-328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055730#comment-14055730
] 

Zhiming Guo commented on STORM-328:
-----------------------------------

Hi there, I'm a newbie and I want to solve this newbie bug. I have read the related codes.
Is it a real bug? 
Should I just simply add a new Utils.getFloat? Or I add Utils.getFloat and Utils.getNumber
then replace all getInt with getNumber?

I ran 
{code}
egrep -ir --include=*.{java,clj} "getInt\(" . | less
{code}
and I got output: 
{code}
./backtype/storm/drpc/DRPCSpout.java:            int port = Utils.getInt(conf.get(Config.DRPC_INVOCATIONS_PORT));
./backtype/storm/drpc/ReturnResults.java:            final int port = Utils.getInt(retMap.get("port"));
./backtype/storm/messaging/netty/Client.java:        buffer_size = Utils.getInt(storm_conf.get(Config.STORM_MESSAGING_NETTY_BUFFER_SIZE));
./backtype/storm/messaging/netty/Client.java:        max_retries = Math.min(30, Utils.getInt(storm_conf.get(Config.STORM_MESSAGING_NETTY_MAX_RETRIES)));
./backtype/storm/messaging/netty/Client.java:        base_sleep_ms = Utils.getInt(storm_conf.get(Config.STORM_MESSAGING_NETTY_MIN_SLEEP_MS));
./backtype/storm/messaging/netty/Client.java:        max_sleep_ms = Utils.getInt(storm_conf.get(Config.STORM_MESSAGING_NETTY_MAX_SLEEP_MS));
./backtype/storm/messaging/netty/Client.java:        this.messageBatchSize = Utils.getInt(storm_conf.get(Config.STORM_NETTY_MESSAGE_BATCH_SIZE),
262144);
./backtype/storm/messaging/netty/Client.java:        flushCheckInterval = Utils.getInt(storm_conf.get(Config.STORM_NETTY_FLUSH_CHECK_INTERVAL_MS),
10); // default 10 ms
./backtype/storm/messaging/netty/Context.java:        int maxWorkers = Utils.getInt(storm_conf.get(Config.STORM_MESSAGING_NETTY_CLIENT_WORKER_THREADS));
./backtype/storm/messaging/netty/Context.java:        int otherWorkers = Utils.getInt(storm_conf.get(Config.TOPOLOGY_WORKERS),
1) - 1;
./backtype/storm/messaging/netty/Server.java:        queueCount = Utils.getInt(storm_conf.get(Config.WORKER_RECEIVER_THREAD_COUNT),
1);
./backtype/storm/messaging/netty/Server.java:        int buffer_size = Utils.getInt(storm_conf.get(Config.STORM_MESSAGING_NETTY_BUFFER_SIZE));
./backtype/storm/messaging/netty/Server.java:        int maxWorkers = Utils.getInt(storm_conf.get(Config.STORM_MESSAGING_NETTY_SERVER_WORKER_THREADS));
./backtype/storm/task/GeneralTopologyContext.java:        Integer max = Utils.getInt(_stormConf.get(Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS));
./backtype/storm/task/GeneralTopologyContext.java:                    max = Math.max(Utils.getInt(comp),
max);
./backtype/storm/testing/MemoryTransactionalSpout.java:            else _maxSpoutPending =
Utils.getInt(c);
./backtype/storm/testing/OpaqueMemoryTransactionalSpout.java:            else _maxSpoutPending
= Utils.getInt(c);
./backtype/storm/transactional/TransactionalSpoutCoordinator.java:            _maxTransactionActive
= Utils.getInt(active);
./backtype/storm/utils/NimbusClient.java:            int nimbusPort = Utils.getInt(conf.get(Config.NIMBUS_THRIFT_PORT));
./backtype/storm/utils/Utils.java:    public static Integer getInt(Object o) {
./backtype/storm/utils/Utils.java:      Integer result = getInt(o, null);
./backtype/storm/utils/Utils.java:    public static Integer getInt(Object o, Integer defaultValue)
{
./backtype/storm/utils/Utils.java:            serverPorts.add(zkServer + ":" + Utils.getInt(port));
./backtype/storm/utils/Utils.java:                .connectionTimeoutMs(Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT)))
./backtype/storm/utils/Utils.java:                .sessionTimeoutMs(Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_SESSION_TIMEOUT)))
./backtype/storm/utils/Utils.java:                            Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_RETRY_INTERVAL)),
./backtype/storm/utils/Utils.java:                            Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_RETRY_TIMES)),
./backtype/storm/utils/Utils.java:                            Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_RETRY_INTERVAL_CEILING))));
./storm/trident/drpc/ReturnResultsReducer.java:            final int port = Utils.getInt(retMap.get("port"));
{code}

> Config.java and Utils.get{Int,Long} are not in sync for floating point
> ----------------------------------------------------------------------
>
>                 Key: STORM-328
>                 URL: https://issues.apache.org/jira/browse/STORM-328
>             Project: Apache Storm (Incubating)
>          Issue Type: Bug
>    Affects Versions: 0.9.2-incubating, 0.9.1-incubating, 0.9.0.1
>            Reporter: Robert Joseph Evans
>              Labels: newbie
>
> For most numeric configuration values Config.java has a type of Number listed, but the
values are parsed using Utils.getInt or Utils.getLong, which means if someone gave a floating
point number it would pass the ConfigValidation, but would blow up when it is used.  This
is most critical for values that Nimbus reads, but would be good to have them consistent everywhere.
> We should also check that getInt works properly for numbers that are larger then would
fit in an integer.
> Either we need to update Utils to be more lenient when looking at doubles, or we need
to make Config more strict, which may be difficult with the YAML and JSON parsing that happens,
where ints are often converted into longs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message