hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: pagerank failed with no particular error log info
Date Fri, 19 Oct 2012 11:40:41 GMT
Please use Hadoop RPC instead of Avro.

On Fri, Oct 19, 2012 at 4:37 PM, Sandy Ding <sandy.dingxin@gmail.com> wrote:
> As follows, still no specific error info
>
> STARTUP_MSG: Starting BSPMaster
> STARTUP_MSG:   host = debian-sandy/127.0.1.1
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 1.0.0
> STARTUP_MSG:   build =
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
> 1214675; compiled by 'hortonfo' on Fri Dec 16 20:01:27 UTC 2011
> ************************************************************/
> 2012-10-19 13:51:21,855 INFO org.apache.hama.bsp.BSPMaster: RPC BSPMaster:
> host localhost port 40000
> 2012-10-19 13:51:22,078 INFO org.apache.hadoop.ipc.Server: Starting
> SocketReader
> 2012-10-19 13:51:22,148 INFO org.mortbay.log: Logging to
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
> 2012-10-19 13:51:22,188 INFO org.apache.hama.http.HttpServer: Port returned
> by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening
> the listener on 40013
> 2012-10-19 13:51:22,189 INFO org.apache.hama.http.HttpServer:
> listener.getLocalPort() returned 40013
> webServer.getConnectors()[0].getLocalPort() returned 40013
> 2012-10-19 13:51:22,189 INFO org.apache.hama.http.HttpServer: Jetty bound
> to port 40013
> 2012-10-19 13:51:22,189 INFO org.mortbay.log: jetty-6.1.14
> 2012-10-19 13:51:22,416 INFO org.mortbay.log: Started
> SelectChannelConnector@localhost:40013
> 2012-10-19 13:51:22,565 INFO org.apache.hama.bsp.BSPMaster: Cleaning up the
> system directory
> 2012-10-19 13:51:22,565 INFO org.apache.hama.bsp.BSPMaster:
> hdfs://localhost:54310/tmp/bsp/system
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:host.name=debian-sandy.localdomain
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.version=1.7.0_07
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.vendor=Oracle Corporation
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/sun-jdk1.7/jre
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.class.path=/home/sandy/workspace/hama-trunk/bin/../conf:/usr/lib/sun-jdk1.7/lib/tools.jar:/home/sandy/workspace/hama-trunk/bin/../core/target/classes:/home/sandy/workspace/hama-trunk/bin/../graph/target/classes:/home/sandy/workspace/hama-trunk/bin/../hama-**.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-launcher-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-ipc-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-configuration-1.7.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-lang-2.6.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-logging-1.1.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-math3-3.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/guava-10.0.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-core-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-test-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-core-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-mapper-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-annotations-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-util-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-api-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/junit-4.8.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/log4j-1.2.16.jar:/home/sandy/workspace/hama-trunk/bin/../lib/netty-3.2.6.Final.jar:/home/sandy/workspace/hama-trunk/bin/../lib/servlet-api-6.0.32.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-api-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/snappy-java-1.0.4.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/zookeeper-3.3.3.jar
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=/tmp
> 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.compiler=<NA>
> 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:os.name=Linux
> 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:os.arch=amd64
> 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:os.version=2.6.32-5-amd64
> 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:user.name=sandy
> 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:user.home=/home/sandy
> 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:user.dir=/home/sandy/workspace/hama-trunk
> 2012-10-19 13:51:22,810 INFO org.apache.zookeeper.ZooKeeper: Initiating
> client connection, connectString=localhost:21810 sessionTimeout=24000000
> watcher=org.apache.hama.bsp.BSPMaster@3c75339f
> 2012-10-19 13:51:22,819 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server localhost/127.0.0.1:21810
> 2012-10-19 13:51:22,819 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to localhost/127.0.0.1:21810, initiating session
> 2012-10-19 13:51:22,837 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server localhost/127.0.0.1:21810, sessionid =
> 0x13a77936b710000, negotiated timeout = 24000000
> 2012-10-19 13:51:22,870 INFO org.apache.hadoop.ipc.Server: IPC Server
> Responder: starting
> 2012-10-19 13:51:22,870 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 40000: starting
> 2012-10-19 13:51:22,871 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 0 on 40000: starting
> 2012-10-19 13:51:22,873 INFO org.apache.hama.bsp.BSPMaster: Starting RUNNING
> 2012-10-19 13:51:31,107 INFO org.apache.hama.bsp.BSPMaster:
> groomd_debian-sandy.localdomain_50000 is added.
> 2012-10-19 13:51:31,753 WARN org.apache.hadoop.util.NativeCodeLoader:
> Unable to load native-hadoop library for your platform... using
> builtin-java classes where applicable
> 2012-10-19 13:51:31,910 INFO org.apache.hama.bsp.JobInProgress: num
> BSPTasks: 3
> 2012-10-19 13:51:31,934 INFO org.apache.hama.bsp.JobInProgress: Job is
> initialized.
> 2012-10-19 13:51:33,176 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0001_000001_0' has failed.
> 2012-10-19 13:51:33,176 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0001_000001_0' has failed.
> 2012-10-19 13:51:33,177 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0001_000001' has failed.
> 2012-10-19 13:51:33,326 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 13:51:33,332 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0001_000000_0' has failed.
> 2012-10-19 13:51:33,332 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0001_000000' has failed.
> 2012-10-19 13:51:33,338 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 13:51:33,348 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0001_000002_0' has failed.
> 2012-10-19 13:51:33,348 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0001_000002' has failed.
> 2012-10-19 13:51:33,349 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 13:52:07,216 INFO org.apache.hama.bsp.JobInProgress: num
> BSPTasks: 3
> 2012-10-19 13:52:07,219 INFO org.apache.hama.bsp.JobInProgress: Job is
> initialized.
> 2012-10-19 13:52:37,322 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0002_000002_0' has failed.
> 2012-10-19 13:52:37,334 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0002_000002' has failed.
> 2012-10-19 13:52:38,040 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 13:52:38,507 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0002_000001_0' has failed.
> 2012-10-19 13:52:38,507 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0002_000001' has failed.
> 2012-10-19 13:52:39,276 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 14:01:32,449 INFO org.apache.hama.bsp.JobInProgress: num
> BSPTasks: 3
> 2012-10-19 14:01:32,733 INFO org.apache.hama.bsp.JobInProgress: Job is
> initialized.
> 2012-10-19 14:01:42,770 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0003_000002_0' has failed.
> 2012-10-19 14:01:42,770 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0003_000002' has failed.
> 2012-10-19 14:01:43,047 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 14:01:43,137 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0003_000000_0' has failed.
> 2012-10-19 14:01:43,137 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0003_000000' has failed.
> 2012-10-19 14:01:43,146 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 14:01:43,159 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0003_000001_0' has failed.
> 2012-10-19 14:01:43,159 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0003_000001' has failed.
> 2012-10-19 14:01:43,167 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 14:07:11,209 INFO org.apache.hama.bsp.JobInProgress: num
> BSPTasks: 3
> 2012-10-19 14:07:11,311 INFO org.apache.hama.bsp.JobInProgress: Job is
> initialized.
> 2012-10-19 14:07:23,655 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0004_000002_0' has failed.
> 2012-10-19 14:07:23,670 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0004_000002' has failed.
> 2012-10-19 14:07:23,706 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 14:07:23,728 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0004_000000_0' has failed.
> 2012-10-19 14:07:23,728 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0004_000000' has failed.
> 2012-10-19 14:07:23,729 INFO org.apache.hama.bsp.JobInProgress: Job failed.
> 2012-10-19 14:07:23,737 INFO org.apache.hama.bsp.JobInProgress: Taskid
> 'attempt_201210191351_0004_000001_0' has failed.
> 2012-10-19 14:07:23,737 INFO org.apache.hama.bsp.TaskInProgress: Task
> 'task_201210191351_0004_000001' has failed.
> 2012-10-19 14:07:23,738 INFO org.apache.hama.bsp.JobInProgress: Job failed.
>
>
> 2012/10/19 Thomas Jungblut <thomas.jungblut@gmail.com>
>
>> Can you show the bsp master log?
>>
>> 2012/10/19 Sandy Ding <sandy.dingxin@gmail.com>
>>
>> > Hi, all
>> >
>> > I run pagerank under the pseudo distributed mode with the web-google.txt
>> > input.
>> > But the job kept failed without error log infomation. The tasklog is as
>> > follows:
>> > What might be wrong?
>> >
>> >
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27
>> GMT
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client environment:host.name
>> > =debian-sandy.localdomain
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:java.version=1.7.0_07
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:java.vendor=Oracle Corporation
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:java.home=/usr/lib/sun-jdk1.7/jre
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> >
>> >
>> environment:java.class.path=/home/sandy/workspace/hama-trunk/bin/../conf:/usr/lib/sun-jdk1.7/lib/tools.jar:/home/sandy/workspace/hama-trunk/bin/../core/target/classes:/home/sandy/workspace/hama-trunk/bin/../graph/target/classes:/home/sandy/workspace/hama-trunk/bin/../hama-**.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-launcher-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-ipc-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-configuration-1.7.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-lang-2.6.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-logging-1.1.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-math3-3.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/guava-10.0.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-core-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-test-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-core-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-mapper-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-annotations-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-util-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-api-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/junit-4.8.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/log4j-1.2.16.jar:/home/sandy/workspace/hama-trunk/bin/../lib/netty-3.2.6.Final.jar:/home/sandy/workspace/hama-trunk/bin/../lib/servlet-api-6.0.32.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-api-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/snappy-java-1.0.4.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/zookeeper-3.3.3.jar::/tmp/bsp-local/groomServer/attempt_201210191351_0003_000001_0/work/classes:/tmp/bsp-local/groomServer/attempt_201210191351_0003_000001_0/work
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> >
>> >
>> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:java.io.tmpdir=/tmp
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:java.compiler=<NA>
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client environment:os.name
>> > =Linux
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:os.arch=amd64
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:os.version=2.6.32-5-amd64
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client environment:user.name
>> > =sandy
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> > environment:user.home=/home/sandy
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client
>> >
>> >
>> environment:user.dir=/tmp/bsp-local/groomServer/attempt_201210191351_0003_000001_0/work
>> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Initiating client connection,
>> > connectString=localhost:21810 sessionTimeout=24000000
>> > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImpl@67af0d15
>> > 12/10/19 14:01:34 INFO zookeeper.ClientCnxn: Opening socket connection to
>> > server localhost/127.0.0.1:21810
>> > 12/10/19 14:01:34 INFO zookeeper.ClientCnxn: Socket connection
>> established
>> > to localhost/127.0.0.1:21810, initiating session
>> > 12/10/19 14:01:34 INFO sync.ZooKeeperSyncClientImpl: Start connecting to
>> > Zookeeper! At debian-sandy.localdomain/127.0.1.1:61001
>> > 12/10/19 14:01:34 INFO zookeeper.ClientCnxn: Session establishment
>> complete
>> > on server localhost/127.0.0.1:21810, sessionid = 0x13a77936b710005,
>> > negotiated timeout = 24000000
>> > 12/10/19 14:01:35 INFO bsp.BSPTask: [DX ping]: ping period is 30000going
>> to
>> > ping groom
>> > 12/10/19 14:01:35 ERROR bsp.BSPTask: Started pinging to groom
>> > 12/10/19 14:01:35 INFO graph.GraphJobRunner: [DX]: starting to load
>> > vertices
>> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: Connecting to
>> > debian-sandy.localdomain/127.0.1.1:61003
>> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c] OPEN
>> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, /
>> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003] BOUND: /
>> > 127.0.1.1:55129
>> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, /
>> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003] CONNECTED:
>> > debian-sandy.localdomain/127.0.1.1:61003
>> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, /
>> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003]
>> > INTEREST_CHANGED
>> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, /
>> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003]
>> > INTEREST_CHANGED
>> >
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message