giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patel, Nishant - Time Inc India <>" <>
Subject Re: Graph job stuck after map 100% completed
Date Thu, 18 Feb 2016 05:02:03 GMT
Hi Khaled,

Thank you for the response.

When I executed it on my local system it failed due to insufficient memory.

I am new to EMR and due to some configuration issue it was not giving error message. Job was
not completed and I had to manually kill job.

Now I have question. I need to launch EMR for my Giraph job. How can I know or calculate how
much memory and nodes required for my job?

I know my data in terms of disk size. Do I need to keep adding node till Memory issues goes
or is there a way to know how much nodes I will be required?

I am new to Giraph so still learning how to get all these details.


From: "<>" <<>>
Reply-To: "<>" <<>>
Date: Wednesday, February 17, 2016 at 6:57 PM
To: "Patel, Nishant - Time Inc India <<>>"
Subject: Re: Graph job stuck after map 100% completed

Hi Ni???shant,

It looks like there is no enough memory to hold the data. How much aggregate memory available
in your cluster?


Sent from my BlackBerry 10 smartphone.
From: Patel, Nishant - Time Inc India <<>>
Sent: Wednesday, February 17, 2016 12:14 AM
Reply To:<>
Subject: Graph job stuck after map 100% completed

Hi All,

I am running Giraph job with around 250 MB of compressed data. I am using EMR with one master
and two data nodes. My input directory contain 18 files.

If I launch 18 jobs for all files separately it works fine. I am not doing any computation.
In first super step itself all vertex vote for halt.

I am not getting any error. Job keep running for hours and does not produce anything.

Appreciate any help.


Please be alert for any emails that may ask you for login information or directs you to login
via a link. If you believe this message is a phish or aren't sure whether this message is
trustworthy, please send the original message as an attachment to '<mailto:'>'.

View raw message