giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gianmarco De Francisci Morales <g...@apache.org>
Subject Re: Review Request: Out-of-core messages
Date Fri, 03 Aug 2012 11:03:36 GMT
Hi,

>Are you saying that out-of-core is faster that hitting memory boundaries
> >(i.e. GC)?  It is a bit tough to imagine that out-of-core beats in-core
> >=).
>
> That's the only explanation I could think of, honestly it sounds wrong to
> me too. But those are the results I keep getting. If someone has a better
> one I'd love to hear it :-)


I am not surprised.
Streaming sequentially from a disk is faster than random reading from
memory [1].
Add the GC overhead, and you get an explanation for your results.

[1] The Pathologies of Big Data, http://queue.acm.org/detail.cfm?id=1563874

Cheers,
--
Gianmarco

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message