giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ameya Vilankar <>
Subject Running Giraph with out of core messages and combiners at the same time
Date Mon, 28 Oct 2013 18:46:03 GMT
I have the following questions:
1. Can we use the following parameters:
-D giraph.useOutOfCoreMessages=true and -D
giraph.useBigDataIOForMessages=true at the same time. Can we use the unsafe
serialization and the out of core messaging at the same time?

2. Also whenever I use the -D giraph.useOutOfCoreMessages=true, the
combiners don't run while sending the messages. I need the combiners to run
in all the other steps where I am using them except for the similarity
computation step where I cannot combine the messages being sent. In my code
I just read the first message received in the message list because I know
that running the combiners will result in only one message being received
by all the vertices. But, running out of core causes the combiners not to
run and so I read only the first message and do the processing which is

Is there any way to run the program with out of core messages and combiners
working on those messages.
I have to use run with out of core messages enabled because in the second
superstep I cannot combine the messages and I want to run this on 100
Million users and the 100 Million items.

Please Help me out.
Thanks in advance.
Ameya Vilankar
Software Engineer Intern

View raw message