hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: HAMA PageRank Experiments Failed
Date Thu, 18 Oct 2012 11:13:03 GMT
FYI, I've created a issue for this.
https://issues.apache.org/jira/browse/HAMA-655

On Thu, Oct 18, 2012 at 7:09 PM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> I guess you should check whether vertex reader reads correctly.
>
> See http://wiki.apache.org/hama/WriteHamaGraphFile
>
> On Thu, Oct 18, 2012 at 4:26 PM, shuo wang <ecisp.wangshuo@gmail.com> wrote:
>> Hi,
>>
>> I have done the PageRank experiment on HAMA,but it  failed.I used the data
>> of Stanford University's SNAP.The data types like this:
>>
>>     1   2
>>     1   4
>>     2   78
>>     2   1999
>>
>> 1.When I use the Stanford data directly,the output is:
>>
>>    [hadoop@namenode hama]$ bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar
>> pagerank PageRank/web-Stanford.txt PageRank/4.txt
>>    12/10/18 09:14:10 INFO bsp.FileInputFormat: Total input paths to process
>> : 1
>>    12/10/18 09:14:10 INFO bsp.FileInputFormat: Total # of splits: 45
>>    12/10/18 09:14:11 INFO bsp.BSPJobClient: Running job:
>> job_201210180912_0001
>>    12/10/18 09:14:14 INFO bsp.BSPJobClient: Current supersteps number: 0
>>    12/10/18 09:14:26 INFO bsp.BSPJobClient: Current supersteps number: 2
>>    12/10/18 09:14:35 INFO bsp.BSPJobClient: Current supersteps number: 3
>>    12/10/18 09:14:38 INFO bsp.BSPJobClient: Current supersteps number: 5
>>    12/10/18 09:14:41 INFO bsp.BSPJobClient: Current supersteps number: 9
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient: Current supersteps number: 11
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient: The total number of supersteps:
>> 11
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient: Counters: 8
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:
>> org.apache.hama.bsp.JobInProgress$JobCounter
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:     LAUNCHED_TASKS=45
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:
>> org.apache.hama.bsp.BSPPeerImpl$PeerCounter
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:     SUPERSTEPS=11
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:     SUPERSTEP_SUM=495
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:     TIME_IN_SYNC_MS=242575
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:     IO_BYTES_READ=32888164
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_SENT=6320240
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:     TASK_INPUT_RECORDS=2312497
>>    12/10/18 09:14:44 INFO bsp.BSPJobClient:
>> TOTAL_MESSAGES_RECEIVED=3160120
>>    Job Finished in 34.303 seconds
>>
>>    II think it fails because it just has 11 supersteps and the time is so
>> short,but I don't know the reason.
>>
>> 2.When I write a small program to process the data to the following type:
>>
>>    1   2   4
>>    2   78    1999
>>
>>    the output is:
>>
>>    [hadoop@namenode hama]$ bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar
>> pagerank HamaPR out
>>    12/10/18 09:35:13 INFO bsp.FileInputFormat: Total input paths to process
>> : 1
>>    12/10/18 09:35:13 INFO bsp.FileInputFormat: Total # of splits: 45
>>    12/10/18 09:35:13 INFO bsp.BSPJobClient: Running job:
>> job_201210180932_0002
>>    12/10/18 09:35:16 INFO bsp.BSPJobClient: Current supersteps number: 0
>>    12/10/18 09:35:25 INFO bsp.BSPJobClient: Current supersteps number: 2
>>    12/10/18 09:35:40 INFO bsp.BSPJobClient: Current supersteps number: 0
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
>> bsp.BSPPeerImpl: Moving to local cache files:  INITIALLY IT WAS: null
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
>> sync.ZKSyncClient: Initializing ZK Sync Client
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At datanode07/
>> 192.168.1.217:61002
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
>> Starting SocketReader
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
>> IPC Server Responder: starting
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
>> IPC Server listener on 61002: starting
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
>> message.HadoopMessageManagerImpl:  BSPPeer address:datanode07 port:61002
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
>> IPC Server handler 0 on 61002: starting
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 ERROR bsp.BSPTask:
>> Error running bsp setup and bsp function.
>>    attempt_201210180932_0002_000015_0: java.lang.NullPointerException
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> Stopping server on 61002
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> IPC Server handler 0 on 61002: exiting
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> Stopping IPC Server listener on 61002
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> Stopping IPC Server Responder
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO
>> metrics.RpcInstrumentation: shut down
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 ERROR bsp.BSPTask:
>> Shutting down ping service.
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 FATAL
>> bsp.GroomServer: Error running child
>>    attempt_201210180932_0002_000015_0: java.lang.NullPointerException
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> Stopping server on 61002
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> IPC Server handler 0 on 61002: exiting
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> Stopping IPC Server listener on 61002
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
>> Stopping IPC Server Responder
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO
>> metrics.RpcInstrumentation: shut down
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 ERROR bsp.BSPTask:
>> Shutting down ping service.
>>    attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 FATAL
>> bsp.GroomServer: Error running child
>>    attempt_201210180932_0002_000015_0: java.lang.NullPointerException
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
>>    attempt_201210180932_0002_000015_0: java.lang.NullPointerException
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
>>    attempt_201210180932_0002_000015_0:     at
>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
>>    12/10/18 09:35:40 INFO bsp.BSPJobClient: Job failed.
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message