kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KYLIN-3398) Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum
Date Tue, 05 Jun 2018 19:04:00 GMT
Ted Yu created KYLIN-3398:
-----------------------------

             Summary: Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum
                 Key: KYLIN-3398
                 URL: https://issues.apache.org/jira/browse/KYLIN-3398
             Project: Kylin
          Issue Type: Bug
            Reporter: Ted Yu


There're two issues with the following code:
{code}
    private int calculateShardNum(KylinConfig kylinConfig, long dataSize) {
        long shardSize = kylinConfig.getExtTableSnapshotShardingMB() * 1024 * 1024;
        return dataSize < shardSize ? 1 : (int) (Math.ceil(dataSize / shardSize));
{code}
getExtTableSnapshotShardingMB returns an int. The multiplication is done using 32-bit arithmetic,
and then used in a context that expects an expression of type "long".

Second, Math.ceil expects a double. The integer division would lose some precision.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message