hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "狼侠" <pengchen0...@foxmail.com>
Subject 回复: 回复: sssp experiment
Date Mon, 24 Dec 2012 04:59:21 GMT
But when I use 1 million vertices and each vertex has 100 edges,the tasks is 45.It also can
not run successful.What is the maximum input data that the hama can run? 发送 




------------------ 原始邮件 ------------------
发件人: "Edward J. Yoon"; 
发送时间: 2012年12月24日(星期一) 中午12:31
收件人: "dev"; 
主题: Re: 回复: sssp experiment



> The input graph has 10 million vertex.Each vertex has 100 edges.

10 million vertices and 10 billion edges? The input is too large.
you'll need at least 2 Racks.

On Mon, Dec 24, 2012 at 11:00 AM, 狼侠 <pengchen0525@foxmail.com> wrote:
> 12/12/24 09:28:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
> 12/12/24 09:28:50 WARN snappy.LoadSnappy: Snappy native library not loaded
> 12/12/24 09:28:51 INFO sync.ZKSyncClient: Initializing ZK Sync Client
> 12/12/24 09:28:51 INFO sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
/192.168.1.211:61004
> 12/12/24 09:28:51 ERROR sync.ZooKeeperSyncClientImpl: org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /bsp/job_201212240927_0001/peers
> 12/12/24 09:28:52 INFO ipc.Server: Starting SocketReader
> 12/12/24 09:28:52 INFO ipc.Server: IPC Server Responder: starting
> 12/12/24 09:28:52 INFO message.HadoopMessageManagerImpl:  BSPPeer address:datanode01
port:61004
> 12/12/24 09:28:52 INFO ipc.Server: IPC Server listener on 61004: starting
> 12/12/24 09:28:52 INFO ipc.Server: IPC Server handler 0 on 61004: starting
> 12/12/24 09:37:39 ERROR security.UserGroupInformation: PriviledgedActionException as:hadoop
cause:java.io.IOException: java.lang.OutOfMemoryError: Java heap space
>
>
>
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Suraj Menon"<menonsuraj5@gmail.com>;
> 发送时间: 2012年12月24日(星期一) 凌晨0:20
> 收件人: "dev"<dev@hama.apache.org>;
>
> 主题: Re: sssp experiment
>
>
>
> Please share the error you observe in the logs area -
> $HAMA_HOME/logs/tasklogs/job_201212231344_0002/*.log
>
> -Suraj
>
> On Sun, Dec 23, 2012 at 3:24 AM, 狼侠 <pengchen0525@foxmail.com> wrote:
>
>> Hi,      There is something wrong in my sssp experiment.My hama cluster
>> has 9 nodes.The input graph has 10 million vertex.Each vertex has 100
>> edges.And the number of the tasks is 36.But the job failed.What happened??
>> The error message as follows:
>> [hadoop@namenode hama]$ bin/hama jar hama-examples-0.6.0.jar sssp 0
>> sssp_graph/10m_100e_36s sssp_graph_output/10m_100e_36s 36
>> 12/12/23 14:49:30 INFO bsp.FileInputFormat: Total input paths to process :
>> 36
>> 12/12/23 14:49:31 INFO bsp.BSPJobClient: Running job: job_201212231344_0002
>> 12/12/23 14:49:34 INFO bsp.BSPJobClient: Current supersteps number: 0
>> 12/12/23 14:49:49 INFO bsp.BSPJobClient: Current supersteps number: 1
>> 12/12/23 14:49:52 INFO bsp.BSPJobClient: Current supersteps number: 2
>> 12/12/23 14:50:55 INFO bsp.BSPJobClient: Current supersteps number: 3
>> 12/12/23 14:51:55 INFO bsp.BSPJobClient: Current supersteps number: 4
>> 12/12/23 14:52:01 INFO bsp.BSPJobClient: Current supersteps number: 5
>> 12/12/23 14:52:04 INFO bsp.BSPJobClient: Current supersteps number: 6
>> 12/12/23 14:52:07 INFO bsp.BSPJobClient: Current supersteps number: 7
>> 12/12/23 14:52:10 INFO bsp.BSPJobClient: Current supersteps number: 8
>> 12/12/23 14:53:34 INFO bsp.BSPJobClient: Current supersteps number: 9
>> 12/12/23 14:56:38 INFO bsp.BSPJobClient: Job failed.
>> [hadoop@namenode hama]$



-- 
Best Regards, Edward J. Yoon
@eddieyoon
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message