spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ma...@apache.org
Subject [3/3] git commit: Merge pull request #204 from rxin/hash
Date Tue, 26 Nov 2013 04:49:01 GMT
Merge pull request #204 from rxin/hash

OpenHashSet fixes

Incorporated ideas from pull request #200.
- Use Murmur Hash 3 finalization step to scramble the bits of HashCode
  instead of the simpler version in java.util.HashMap; the latter one
  had trouble with ranges of consecutive integers. Murmur Hash 3 is used
  by fastutil.
- Don't check keys for equality when re-inserting due to growing the
  table; the keys will already be unique.
- Remember the grow threshold instead of recomputing it on each insert

Also added unit tests for size estimation for specialized hash sets and maps.


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/0e2109dd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/0e2109dd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/0e2109dd

Branch: refs/heads/master
Commit: 0e2109ddb2f27d8a6a9f125206674273b03d1f5e
Parents: c46067f 466fd06
Author: Matei Zaharia <matei@eecs.berkeley.edu>
Authored: Mon Nov 25 20:48:37 2013 -0800
Committer: Matei Zaharia <matei@eecs.berkeley.edu>
Committed: Mon Nov 25 20:48:37 2013 -0800

----------------------------------------------------------------------
 .../spark/util/collection/OpenHashSet.scala     | 107 ++++++++++---------
 .../util/collection/OpenHashMapSuite.scala      |  16 ++-
 .../util/collection/OpenHashSetSuite.scala      |  20 +++-
 .../PrimitiveKeyOpenHashMapSuite.scala          | 102 ++++++++++++++++++
 .../PrimitiveKeyOpenHashSetSuite.scala          |  90 ----------------
 5 files changed, 192 insertions(+), 143 deletions(-)
----------------------------------------------------------------------



Mime
View raw message