giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Kumar <toga...@gmail.com>
Subject Out of memory error in Graph on hadoop1.2.1
Date Fri, 04 Apr 2014 13:37:31 GMT
Hi

while executing triangle count progrm over graph in a cluster of 8 server
machines  i am getting out of memory exception.My data size is of 1GB
In my conf/hadoop-env.sh file i have updated the heapsize

export HADOOP_HEAPSIZE=6000

even after that i am getting the below exception

Can someone help ?


     at org\.apache\.giraph\.graph\.GraphMapper\.run(GraphMapper\.java:102)
    at org\.apache\.hadoop\.mapred\.MapTask\.runNewMapper(MapTask\.java:764)
    at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:364)
    at org\.apache\.hadoop\.mapred\.Child$4\.run(Child\.java:255)
    at java\.security\.AccessController\.doPrivileged(Native Method)
    at javax\.security\.auth\.Subject\.doAs(Subject\.java:396)
    at
org\.apache\.hadoop\.security\.UserGroupInformation\.doAs(UserGroupInformation\.java:1190)
    at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:249)
Caused by: java\.lang\.IllegalStateException: waitFor: ExecutionException
occurred while waiting for
org\.apache\.giraph\.utils\.ProgressableUtils$FutureWaitable@2a0850f8
    at
org\.apache\.giraph\.utils\.ProgressableUtils\.waitFor(ProgressableUtils\.java:151)
    at
org\.apache\.giraph\.utils\.ProgressableUtils\.waitForever(ProgressableUtils\.java:111)
    at
org\.apache\.giraph\.utils\.ProgressableUtils\.getFutureResult(ProgressableUtils\.java:73)
    at
org\.apache\.giraph\.utils\.ProgressableUtils\.getResultsWithNCallables(ProgressableUtils\.java:192)
    at
org\.apache\.giraph\.graph\.GraphTaskManager\.processGraphPartitions(GraphTaskManager\.java:753)
    at
org\.apache\.giraph\.graph\.GraphTaskManager\.execute(GraphTaskManager\.java:273)
    at org\.apache\.giraph\.graph\.GraphMapper\.run(GraphMapper\.java:92)
    \.\.\. 7 more
Caused by: java\.util\.concurrent\.ExecutionException:
java\.lang\.OutOfMemoryError: Java heap space
    at
java\.util\.concurrent\.FutureTask$Sync\.innerGet(FutureTask\.java:232)
    at java\.util\.concurrent\.FutureTask\.get(FutureTask\.java:91)
    at
org\.apache\.giraph\.utils\.ProgressableUtils$FutureWaitable\.waitFor(ProgressableUtils\.java:271)
    at
org\.apache\.giraph\.utils\.ProgressableUtils\.waitFor(ProgressableUtils\.java:143)
    \.\.\. 13 more
Caused by: java\.lang\.OutOfMemoryError: Java heap space
    at
org\.apache\.giraph\.utils\.UnsafeByteArrayOutputStream\.ensureSize(UnsafeByteArrayOutputStream\.java:131)
    at
org\.apache\.giraph\.utils\.UnsafeByteArrayOutputStream\.writeLong(UnsafeByteArrayOutputStream\.java:234)
    at org\.apache\.hadoop\.io\.LongWritable\.write(LongWritable\.java:42)
    at
org\.apache\.giraph\.comm\.messages\.ByteArrayMessagesPerVertexStore\.addPartitionMessages(ByteArrayMessagesPerVertexStore\.java:121)
    at
org\.apache\.giraph\.comm\.requests\.SendWorkerMessagesRequest\.doRequest(SendWorkerMessagesRequest\.java:72)
    at
org\.apache\.giraph\.comm\.netty\.NettyWorkerClientRequestProcessor\.doRequest(NettyWorkerClientRequestProcessor\.java:470)
    at
org\.apache\.giraph\.comm\.netty\.NettyWorkerClientRequestProcessor\.sendMessageRequest(NettyWorkerClientRequestProcessor\.java:186)
    at org\.apache\.giraph\.graph\.Vertex\.sendMessage(Vertex\.java:340)
    at
org\.apache\.giraph\.graph\.Vertex\.sendMessageToAllEdges(Vertex\.java:353)
    at
org\.apache\.giraph\.examples\.TriangleCounting\.compute(TriangleCounting\.java:47)
    at
org\.apache\.giraph\.graph\.ComputeCallable\.computePartition(ComputeCallable\.java:234)
    at
org\.apache\.giraph\.graph\.ComputeCallable\.call(ComputeCallable\.java:162)
    at
org\.apache\.giraph\.graph\.ComputeCallable\.call(ComputeCallable\.java:70)
    at
org\.apache\.giraph\.utils\.LogStacktraceCallable\.call(LogStacktraceCallable\.java:51)
    at
java\.util\.concurrent\.FutureTask$Sync\.innerRun(FutureTask\.java:303)
    at java\.util\.concurrent\.FutureTask\.run(FutureTask\.java:138)
    at
java\.util\.concurrent\.ThreadPoolExecutor$Worker\.runTask(ThreadPoolExecutor\.java:895)
    at
java\.util\.concurrent\.ThreadPoolExecutor$Worker\.run(ThreadPoolExecutor\.java:918)
    at java\.lang\.Thread\.run(Thread\.java:662)
,Task attempt_201404040451_0001_m_000001_0 failed to report status for 600
seconds\. Killing!" .

Mime
View raw message