flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1013) ArithmeticException: / by zero in MutableHashTable
Date Tue, 08 Jul 2014 18:26:04 GMT
Till Rohrmann created FLINK-1013:
------------------------------------

             Summary: ArithmeticException: / by zero in MutableHashTable
                 Key: FLINK-1013
                 URL: https://issues.apache.org/jira/browse/FLINK-1013
             Project: Flink
          Issue Type: Bug
            Reporter: Till Rohrmann


I encountered a division by zero exception in the MutableHashTable. It happened when I joined
two datasets of relatively big records (approx. 40-50 MB I think). When joining them the buildTableFromSpilledPartition
method of the MutableHashTable is called. In case that the available buffers are smaller than
the needed number of buffers, the mutable hash table will calculate the bucket count
{code}
bucketCount = (int) (((long) totalBuffersAvailable) * RECORD_TABLE_BYTES / (avgRecordLenPartition
+ RECORD_OVERHEAD_BYTES));
{code}

If the average record length is sufficiently large, then the bucket count will be 0. Initializing
the hash table with a 0 bucket count will cause then the division by 0 exception. I don't
know whether this problem can be mitigated but it should at least throw a meaningful exception
instead of the ArithmeticException.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message