incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gianmarco De Francisci Morales (Commented) (JIRA)" <>
Subject [jira] [Commented] (S4-30) DefaultHasher hashes keys to negative number
Date Sat, 24 Dec 2011 09:56:30 GMT


Gianmarco De Francisci Morales commented on S4-30:

I think that this snippet should do the trick (anyway you lose 1 bit of information because
Java does not have unsigned types and you actually don't care about doing exact arithmetic).
It should be faster than Math.abs() (but I haven't run microbenchmarks on it).

return (int) (rv & 0x7fffffff);
> DefaultHasher hashes keys to negative number
> --------------------------------------------
>                 Key: S4-30
>                 URL:
>             Project: Apache S4
>          Issue Type: Bug
>    Affects Versions: 0.4
>         Environment: All - Windows and Linux
>            Reporter: Quoc Nguyen
>            Priority: Blocker
> DefaultHasher uses HashAlgorithm hashAlgorithm = HashAlgorithm.FNV1_64_HASH; which hashes
key strings such as 118+18233, 118+17360, 118+17258, 118+18147 and 118+18121 and many more
to negative values which the DefaultPartitioner (int partitionId = (int) (hasher.hash(stringValue)
% partitionCount);) tries to partition the key to incorrect partition.
> Workaround:
> None - stream has those keys, they will get dropped because the partitioner cannot correctly

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message