hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shuo wang <ecisp.wangs...@gmail.com>
Subject HAMA PageRank Experiments Failed
Date Thu, 18 Oct 2012 07:26:40 GMT
Hi,

I have done the PageRank experiment on HAMA,but it  failed.I used the data
of Stanford University's SNAP.The data types like this:

    1   2
    1   4
    2   78
    2   1999

1.When I use the Stanford data directly,the output is:

   [hadoop@namenode hama]$ bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar
pagerank PageRank/web-Stanford.txt PageRank/4.txt
   12/10/18 09:14:10 INFO bsp.FileInputFormat: Total input paths to process
: 1
   12/10/18 09:14:10 INFO bsp.FileInputFormat: Total # of splits: 45
   12/10/18 09:14:11 INFO bsp.BSPJobClient: Running job:
job_201210180912_0001
   12/10/18 09:14:14 INFO bsp.BSPJobClient: Current supersteps number: 0
   12/10/18 09:14:26 INFO bsp.BSPJobClient: Current supersteps number: 2
   12/10/18 09:14:35 INFO bsp.BSPJobClient: Current supersteps number: 3
   12/10/18 09:14:38 INFO bsp.BSPJobClient: Current supersteps number: 5
   12/10/18 09:14:41 INFO bsp.BSPJobClient: Current supersteps number: 9
   12/10/18 09:14:44 INFO bsp.BSPJobClient: Current supersteps number: 11
   12/10/18 09:14:44 INFO bsp.BSPJobClient: The total number of supersteps:
11
   12/10/18 09:14:44 INFO bsp.BSPJobClient: Counters: 8
   12/10/18 09:14:44 INFO bsp.BSPJobClient:
org.apache.hama.bsp.JobInProgress$JobCounter
   12/10/18 09:14:44 INFO bsp.BSPJobClient:     LAUNCHED_TASKS=45
   12/10/18 09:14:44 INFO bsp.BSPJobClient:
org.apache.hama.bsp.BSPPeerImpl$PeerCounter
   12/10/18 09:14:44 INFO bsp.BSPJobClient:     SUPERSTEPS=11
   12/10/18 09:14:44 INFO bsp.BSPJobClient:     SUPERSTEP_SUM=495
   12/10/18 09:14:44 INFO bsp.BSPJobClient:     TIME_IN_SYNC_MS=242575
   12/10/18 09:14:44 INFO bsp.BSPJobClient:     IO_BYTES_READ=32888164
   12/10/18 09:14:44 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_SENT=6320240
   12/10/18 09:14:44 INFO bsp.BSPJobClient:     TASK_INPUT_RECORDS=2312497
   12/10/18 09:14:44 INFO bsp.BSPJobClient:
TOTAL_MESSAGES_RECEIVED=3160120
   Job Finished in 34.303 seconds

   II think it fails because it just has 11 supersteps and the time is so
short,but I don't know the reason.

2.When I write a small program to process the data to the following type:

   1   2   4
   2   78    1999

   the output is:

   [hadoop@namenode hama]$ bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar
pagerank HamaPR out
   12/10/18 09:35:13 INFO bsp.FileInputFormat: Total input paths to process
: 1
   12/10/18 09:35:13 INFO bsp.FileInputFormat: Total # of splits: 45
   12/10/18 09:35:13 INFO bsp.BSPJobClient: Running job:
job_201210180932_0002
   12/10/18 09:35:16 INFO bsp.BSPJobClient: Current supersteps number: 0
   12/10/18 09:35:25 INFO bsp.BSPJobClient: Current supersteps number: 2
   12/10/18 09:35:40 INFO bsp.BSPJobClient: Current supersteps number: 0
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
bsp.BSPPeerImpl: Moving to local cache files:  INITIALLY IT WAS: null
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
sync.ZKSyncClient: Initializing ZK Sync Client
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At datanode07/
192.168.1.217:61002
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
Starting SocketReader
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
IPC Server Responder: starting
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
IPC Server listener on 61002: starting
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO
message.HadoopMessageManagerImpl:  BSPPeer address:datanode07 port:61002
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:09 INFO ipc.Server:
IPC Server handler 0 on 61002: starting
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 ERROR bsp.BSPTask:
Error running bsp setup and bsp function.
   attempt_201210180932_0002_000015_0: java.lang.NullPointerException
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
Stopping server on 61002
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
IPC Server handler 0 on 61002: exiting
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
Stopping IPC Server listener on 61002
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
Stopping IPC Server Responder
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO
metrics.RpcInstrumentation: shut down
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 ERROR bsp.BSPTask:
Shutting down ping service.
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 FATAL
bsp.GroomServer: Error running child
   attempt_201210180932_0002_000015_0: java.lang.NullPointerException
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
Stopping server on 61002
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
IPC Server handler 0 on 61002: exiting
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
Stopping IPC Server listener on 61002
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO ipc.Server:
Stopping IPC Server Responder
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 INFO
metrics.RpcInstrumentation: shut down
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 ERROR bsp.BSPTask:
Shutting down ping service.
   attempt_201210180932_0002_000015_0: 12/10/18 09:35:13 FATAL
bsp.GroomServer: Error running child
   attempt_201210180932_0002_000015_0: java.lang.NullPointerException
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
   attempt_201210180932_0002_000015_0: java.lang.NullPointerException
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.partitionMultiSteps(GraphJobRunner.java:517)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:282)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:97)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:165)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
   attempt_201210180932_0002_000015_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1271)
   12/10/18 09:35:40 INFO bsp.BSPJobClient: Job failed.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message