ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Mashenkov <andrey.mashen...@gmail.com>
Subject Optimize integer sets.
Date Sat, 21 Jan 2017 12:26:27 GMT
Hi Guys

Alexei Scherbakov report a ticket few time ago [1]. The solution look
promissing.

Alexei, you wrote that this can save some memory. More over replacing
linked Set structure to array based bit-set
can give a speed-up due to array based structures are cache friendly.

But one thing is not clear for me how we will handle sparsed bit-sets? For
example, if we have 1024 partiotions (as it is by default)
and have much nodes, e.g. 512. In this case, bit-set will occupy 256 bytes
that seem to be more than Set<Integer>.

What do you mean exactly to use bit-set as more compact structure then
Set<Integer> or bit-set with some additional compression?

I would thought, we can use hash-set with open addressing in some cases
like that to get gain of array bases structures over linked structures and
save memory?
For example, we could use such hash-set for small data (64bytes as cache
line size) and use bit-sets for bigger data, if it's possible of course.


Thoughts?

[1] https://issues.apache.org/jira/browse/IGNITE-4554

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message