hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 顾荣 <gurongwal...@gmail.com>
Subject Re: failed to run hama pi example
Date Wed, 12 Sep 2012 08:28:55 GMT
Sry for type error. I mean "To sum up this topic, the hadoop 0.20.2 seems
to be not compatible with hama 0.5 very well, but hadoop 1.0.3 does."

walker


2012/9/12 顾荣 <gurongwalker@gmail.com>

> Hi, Dear Thomas and Edward.
>
> I used hadoop 1.0.3 with hama 0.5 and it works well. My  problem finally
> solved. Thank you very very much. To sum up this topic, the hadoop 0.20.2
> seems to compateiabl with hama 0.5 very well, but hadoop 1.0.3 does.
> I suggest that after test, you can give more infomation about the suitable
> version matches between hadoop and hama under the *Requirements* section
> in thi page http://hama.apache.org/getting_started_with_hama.html .Maybe
> this will benefit to new users.
>
> This is my pi example running log:
> hama jar /home/hadoop/hama_installs/hama-0.5.0/hama-examples-0.5.0.jar pi
> 12/09/12 16:11:53 INFO bsp.BSPJobClient: Running job: job_201209121610_0001
> 12/09/12 16:11:56 INFO bsp.BSPJobClient: Current supersteps number: 1
> 12/09/12 16:11:59 INFO bsp.BSPJobClient: The total number of supersteps: 1
> 12/09/12 16:11:59 INFO bsp.BSPJobClient: Counters: 8
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:
> org.apache.hama.bsp.JobInProgress$JobCounter
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     LAUNCHED_TASKS=9
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:
> org.apache.hama.bsp.BSPPeerImpl$PeerCounter
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     SUPERSTEPS=1
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     COMPRESSED_BYTES_SENT=675
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     SUPERSTEP_SUM=9
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     TIME_IN_SYNC_MS=3319
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     COMPRESSED_BYTES_RECEIVED=675
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_SENT=18
> 12/09/12 16:11:59 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_RECEIVED=9
> Estimated value of PI is    3.135555555555555
>
> It only takes 6 seconds, As far as I know, that's the time cost of the
> hadoop to schedule a job to run (2 heartbeats).Hama is so fast and cute.
> I'll explore more in detail next.
>
> Thank you again.
>
> Regards.
> walker.
>
>
> 2012/9/12 顾荣 <gurongwalker@gmail.com>
>
>> Okay, I'll try Hadoop 1.0.3 with Hama 0.50.
>> Thanks Thomas. I can't wait to explore the Hama world now.
>>
>> walker.
>>
>>
>> 2012/9/12 Thomas Jungblut <thomas.jungblut@gmail.com>
>>
>>> oh okay. I'm not sure if 0.5.0 is really compatible to 20.2, personally I
>>> have installed 1.0.3 and it works fine.
>>> Sorry to let you install all the different versions.
>>>
>>> 2012/9/12 顾荣 <gurongwalker@gmail.com>
>>>
>>> > Thanks Thomas. The HDFS works well. I even put a file from local to it
>>> > successfully. It absolutely left the safemode. The namenode starting
>>> log is
>>> > as below:
>>> >
>>> > 2012-09-12 15:10:39,002 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
>>> > /************************************************************
>>> > STARTUP_MSG: Starting NameNode
>>> > STARTUP_MSG:   host = slave021/192.168.1.21
>>> > STARTUP_MSG:   args = []
>>> > STARTUP_MSG:   version = 0.20.2
>>> > STARTUP_MSG:   build =
>>> > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
>>> > 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
>>> > ************************************************************/
>>> > 2012-09-12 15:10:39,092 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
>>> > Initializing RPC Metrics with hostName=NameNode, port=54310
>>> > 2012-09-12 15:10:39,098 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>> slave021/
>>> > 192.168.1.21:54310
>>> > 2012-09-12 15:10:39,100 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>>> > Initializing JVM Metrics with processName=NameNode, sessionId=null
>>> > 2012-09-12 15:10:39,101 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics:
>>> > Initializing NameNodeMeterics using context
>>> > object:org.apache.hadoop.metrics.spi.NullContext
>>> > 2012-09-12 15:10:39,143 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>>> > fsOwner=hadoop,hadoop_user,wheel
>>> > 2012-09-12 15:10:39,144 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>>> supergroup=supergroup
>>> > 2012-09-12 15:10:39,144 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>>> > isPermissionEnabled=true
>>> > 2012-09-12 15:10:39,150 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics:
>>> > Initializing FSNamesystemMetrics using context
>>> > object:org.apache.hadoop.metrics.spi.NullContext
>>> > 2012-09-12 15:10:39,151 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
>>> > FSNamesystemStatusMBean
>>> > 2012-09-12 15:10:39,177 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage:
>>> > Number of files = 1
>>> > 2012-09-12 15:10:39,181 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage:
>>> > Number of files under construction = 0
>>> > 2012-09-12 15:10:39,181 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage:
>>> > Image file of size 96 loaded in 0 seconds.
>>> > 2012-09-12 15:10:39,181 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage:
>>> > Edits file
>>> > /home/hadoop/gurong/hadoop-0.20.2/hadoop_dir/dfs/name/current/edits of
>>> size
>>> > 4 edits # 0 loaded in 0 seconds.
>>> > 2012-09-12 15:10:39,236 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage:
>>> > Image file of size 96 saved in 0 seconds.
>>> > 2012-09-12 15:10:39,439 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading
>>> > FSImage in 312 msecs
>>> > 2012-09-12 15:10:39,441 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Total number of
>>> blocks
>>> > = 0
>>> > 2012-09-12 15:10:39,441 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of invalid
>>> > blocks = 0
>>> > 2012-09-12 15:10:39,441 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
>>> > under-replicated blocks = 0
>>> > 2012-09-12 15:10:39,441 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
>>> > over-replicated blocks = 0
>>> > 2012-09-12 15:10:39,441 INFO org.apache.hadoop.hdfs.StateChange: STATE*
>>> > Leaving safe mode after 0 secs.
>>> > 2012-09-12 15:10:39,441 INFO org.apache.hadoop.hdfs.StateChange: STATE*
>>> > Network topology has 0 racks and 0 datanodes
>>> > 2012-09-12 15:10:39,441 INFO org.apache.hadoop.hdfs.StateChange: STATE*
>>> > UnderReplicatedBlocks has 0 blocks
>>> > 2012-09-12 15:10:39,554 INFO org.mortbay.log: Logging to
>>> > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>>> > org.mortbay.log.Slf4jLog
>>> > 2012-09-12 15:10:39,603 INFO org.apache.hadoop.http.HttpServer: Port
>>> > returned by webServer.getConnectors()[0].getLocalPort() before open()
>>> is
>>> > -1. Opening the listener on 50070
>>> > 2012-09-12 15:10:39,604 INFO org.apache.hadoop.http.HttpServer:
>>> > listener.getLocalPort() returned 50070
>>> > webServer.getConnectors()[0].getLocalPort() returned 50070
>>> > 2012-09-12 15:10:39,604 INFO org.apache.hadoop.http.HttpServer: Jetty
>>> bound
>>> > to port 50070
>>> > 2012-09-12 15:10:39,604 INFO org.mortbay.log: jetty-6.1.14
>>> > 2012-09-12 15:10:48,662 INFO org.mortbay.log: Started
>>> > SelectChannelConnector@0.0.0.0:50070
>>> > 2012-09-12 15:10:48,663 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at:
>>> > 0.0.0.0:50070
>>> > 2012-09-12 15:10:48,666 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > listener on 54310: starting
>>> > 2012-09-12 15:10:48,667 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > Responder: starting
>>> > 2012-09-12 15:10:48,668 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 0 on 54310: starting
>>> > 2012-09-12 15:10:48,668 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 1 on 54310: starting
>>> > 2012-09-12 15:10:48,668 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 2 on 54310: starting
>>> > 2012-09-12 15:10:48,668 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 3 on 54310: starting
>>> > 2012-09-12 15:10:48,668 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 4 on 54310: starting
>>> > 2012-09-12 15:10:48,669 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 5 on 54310: starting
>>> > 2012-09-12 15:10:48,669 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 6 on 54310: starting
>>> > 2012-09-12 15:10:48,669 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 7 on 54310: starting
>>> > 2012-09-12 15:10:48,669 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 8 on 54310: starting
>>> > 2012-09-12 15:10:48,669 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> > handler 9 on 54310: starting
>>> > 2012-09-12 15:10:48,700 INFO org.apache.hadoop.ipc.Server: Error
>>> register
>>> > getProtocolVersion
>>> > java.lang.IllegalArgumentException: Duplicate
>>> > metricsName:getProtocolVersion
>>> >     at
>>> >
>>> org.apache.hadoop.metrics.util.MetricsRegistry.add(MetricsRegistry.java:53)
>>> >     at
>>> >
>>> >
>>> org.apache.hadoop.metrics.util.MetricsTimeVaryingRate.<init>(MetricsTimeVaryingRate.java:89)
>>> >     at
>>> >
>>> >
>>> org.apache.hadoop.metrics.util.MetricsTimeVaryingRate.<init>(MetricsTimeVaryingRate.java:99)
>>> >     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:523)
>>> >     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
>>> >     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
>>> >     at java.security.AccessController.doPrivileged(Native Method)
>>> >     at javax.security.auth.Subject.doAs(Subject.java:416)
>>> >     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
>>> > 2012-09-12 15:11:05,298 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
>>> > NameSystem.registerDatanode: node registration from
>>> > 192.168.1.21:50010storage
>>> > DS-1416037815-192.168.1.21-50010-1347433865293
>>> > 2012-09-12 15:11:05,300 INFO org.apache.hadoop.net.NetworkTopology:
>>> Adding
>>> > a new node: /default-rack/192.168.1.21:50010
>>> > 2012-09-12 15:11:15,069 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit:
>>> > ugi=webuser,webgroup    ip=/192.168.1.21    cmd=listStatus    src=/
>>> > dst=null    perm=null
>>> > 2012-09-12 15:12:05,034 WARN org.apache.hadoop.ipc.Server: Incorrect
>>> header
>>> > or version mismatch from 192.168.1.21:56281 got version 4 expected
>>> > version 3
>>> > 2012-09-12 15:14:51,535 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit:
>>> > ugi=hadoop,hadoop_user,wheel    ip=/192.168.1.21    cmd=listStatus
>>> > src=/    dst=null    perm=null
>>> > 2012-09-12 15:15:10,158 INFO
>>> > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
>>> > transactions: 2 Total time for transactions(ms): 0Number of
>>> transactions
>>> > batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
>>> >
>>> > I used Hama 0.5 and hadoop 0.20.2. Has somebody test this match can
>>> work
>>> > well?
>>> >
>>> > thanks very much.
>>> >
>>> > walker
>>> >
>>> > 2012/9/12 Thomas Jungblut <thomas.jungblut@gmail.com>
>>> >
>>> > > Still it says:
>>> > > >
>>> > > > " 2012-09-12 14:41:16,218 ERROR org.apache.hama.bsp.BSPMaster:
>>> Can't
>>> > get
>>> > > > connection to Hadoop Namenode! "
>>> > >
>>> > >
>>> > > Can you verify that the namenode is not in safemode and has correctly
>>> > > started up?
>>> > > Have a look into the namenode logs please!
>>> > >
>>> > > 2012/9/12 顾荣 <gurongwalker@gmail.com>
>>> > >
>>> > > > By the way, the fs.default.name is 192.168.1.21:54310. I checked
>>> the
>>> > > HDFS,
>>> > > > it works well. I installed and ran both HDFS and Hama using the
>>> same
>>> > > linux
>>> > > > account.
>>> > > >
>>> > > > 2012/9/12 顾荣 <gurongwalker@gmail.com>
>>> > > >
>>> > > > > Thanks so much, Edward.
>>> > > > >
>>> > > > > I fellowed your suggestion and instanlled a hadoop 0.20.2
>>> instead for
>>> > > > > Hama. However, this time when I start Hama, a fatal happened
and
>>> the
>>> > > > > bspmaster daemon can not start up. The corresponding error
>>> message in
>>> > > the
>>> > > > > baspmaster log file shows as below.
>>> > > > >
>>> > > > > ************************************************************/
>>> > > > > 2012-09-12 14:40:38,238 INFO org.apache.hama.BSPMasterRunner:
>>> > > > STARTUP_MSG:
>>> > > > > /************************************************************
>>> > > > > STARTUP_MSG: Starting BSPMaster
>>> > > > > STARTUP_MSG:   host = slave021/192.168.1.21
>>> > > > > STARTUP_MSG:   args = []
>>> > > > > STARTUP_MSG:   version = 1.0.0
>>> > > > > STARTUP_MSG:   build =
>>> > > > >
>>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0-r
>>> > > > > 1214675; compiled by 'hortonfo' on Fri Dec 16 20:01:27 UTC
2011
>>> > > > > ************************************************************/
>>> > > > > 2012-09-12 14:40:38,414 INFO org.apache.hama.bsp.BSPMaster:
RPC
>>> > > > BSPMaster:
>>> > > > > host slave021 port 40000
>>> > > > > 2012-09-12 14:40:38,502 INFO org.apache.hadoop.ipc.Server:
>>> Starting
>>> > > > > SocketReader
>>> > > > > 2012-09-12 14:40:38,542 INFO org.mortbay.log: Logging to
>>> > > > > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>>> > > > > org.mortbay.log.Slf4jLog
>>> > > > > 2012-09-12 14:40:38,583 INFO org.apache.hama.http.HttpServer:
>>> Port
>>> > > > > returned by webServer.getConnectors()[0].getLocalPort() before
>>> open()
>>> > > is
>>> > > > > -1. Opening the listener on 40013
>>> > > > > 2012-09-12 14:40:38,584 INFO org.apache.hama.http.HttpServer:
>>> > > > > listener.getLocalPort() returned 40013
>>> > > > > webServer.getConnectors()[0].getLocalPort() returned 40013
>>> > > > > 2012-09-12 14:40:38,584 INFO org.apache.hama.http.HttpServer:
>>> Jetty
>>> > > bound
>>> > > > > to port 40013
>>> > > > > 2012-09-12 14:40:38,584 INFO org.mortbay.log: jetty-6.1.14
>>> > > > > 2012-09-12 14:40:38,610 INFO org.mortbay.log: Extract
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> jar:file:/home/hadoop/hama_installs/hama-0.5.0/hama-core-0.5.0.jar!/webapp/bspmaster/
>>> > > > > to /tmp/Jetty_slave021_40013_bspmaster____.1tzgsz/webapp
>>> > > > > 2012-09-12 14:41:16,073 INFO org.mortbay.log: Started
>>> > > > > SelectChannelConnector@slave021:40013
>>> > > > > 2012-09-12 14:41:16,218 ERROR org.apache.hama.bsp.BSPMaster:
>>> Can't
>>> > get
>>> > > > > connection to Hadoop Namenode!
>>> > > > > java.io.IOException: Call to /192.168.1.21:54310 failed on
local
>>> > > > > exception: java.io.EOFException
>>> > > > >     at
>>> org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
>>> > > > >     at org.apache.hadoop.ipc.Client.call(Client.java:1071)
>>> > > > >     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>> > > > >     at $Proxy5.getProtocolVersion(Unknown Source)
>>> > > > >     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
>>> > > > >     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
>>> > > > >     at
>>> > > > >
>>> > org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
>>> > > > >     at
>>> org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
>>> > > > >     at
>>> org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
>>> > > > >     at
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
>>> > > > >     at
>>> > > > >
>>> > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
>>> > > > >     at
>>> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>>> > > > >     at
>>> > org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
>>> > > > >     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
>>> > > > >     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
>>> > > > >     at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:299)
>>> > > > >     at
>>> org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:454)
>>> > > > >     at
>>> org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:449)
>>> > > > >     at
>>> org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:46)
>>> > > > >     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> > > > >     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> > > > >     at
>>> org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:56)
>>> > > > > Caused by: java.io.EOFException
>>> > > > >     at java.io.DataInputStream.readInt(DataInputStream.java:392)
>>> > > > >     at
>>> > > > >
>>> > >
>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
>>> > > > >     at
>>> org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)
>>> > > > > 2012-09-12 14:41:16,222 FATAL org.apache.hama.BSPMasterRunner:
>>> > > > > java.lang.NullPointerException
>>> > > > >     at
>>> org.apache.hama.bsp.BSPMaster.getSystemDir(BSPMaster.java:862)
>>> > > > >     at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:308)
>>> > > > >     at
>>> org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:454)
>>> > > > >     at
>>> org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:449)
>>> > > > >     at
>>> org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:46)
>>> > > > >     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> > > > >     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> > > > >     at
>>> org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:56)
>>> > > > >
>>> > > > > 2012-09-12 14:41:16,223 INFO org.apache.hama.BSPMasterRunner:
>>> > > > > SHUTDOWN_MSG:
>>> > > > > /************************************************************
>>> > > > > SHUTDOWN_MSG: Shutting down BSPMaster at slave021/192.168.1.21
>>> > > > > ************************************************************/
>>> > > > >
>>> > > > > Would please give me some tips again?
>>> > > > >
>>> > > > > Thanks,again.
>>> > > > >
>>> > > > > walker
>>> > > > >
>>> > > > > 2012/9/12 Edward J. Yoon <edwardyoon@apache.org>
>>> > > > >
>>> > > > > Unfortunately we don't support Hadoop secure version yet.
>>> > > > >>
>>> > > > >> Instead of 0.20.205, Please use hadoop non-secure 0.20.2
or
>>> 1.0.3
>>> > > > >> versions.
>>> > > > >>
>>> > > > >> Thanks.
>>> > > > >>
>>> > > > >> On Wed, Sep 12, 2012 at 11:25 AM, 顾荣 <gurongwalker@gmail.com>
>>> > wrote:
>>> > > > >> > Hi,all.
>>> > > > >> >
>>> > > > >> > I set up a hama cluster of 3 nodes and start hama
>>> successfully.
>>> > > > However,
>>> > > > >> > when I run the pi example, the job failed with a
very strange
>>> > > message
>>> > > > as
>>> > > > >> > below.
>>> > > > >> >
>>> > > > >> >  hama jar
>>> > > > /home/hadoop/hama_installs/hama-0.5.0/hama-examples-0.5.0.jar
>>> > > > >> pi
>>> > > > >> > org.apache.hadoop.ipc.RemoteException: java.io.IOException:
>>> > > > >> > java.lang.NoSuchMethodException:
>>> > > > >> >
>>> > > >
>>> org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
>>> > > > >> > org.apache.hadoop.fs.permission.FsPermission,
>>> java.lang.String,
>>> > > > boolean,
>>> > > > >> > boolean, short, long)
>>> > > > >> >     at java.lang.Class.getMethod(Class.java:1605)
>>> > > > >> >     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>>> > > > >> >     at
>>> > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>>> > > > >> >     at
>>> > org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>>> > > > >> >     at java.security.AccessController.doPrivileged(Native
>>> Method)
>>> > > > >> >     at javax.security.auth.Subject.doAs(Subject.java:396)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>>> > > > >> >     at
>>> org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>>> > > > >> >
>>> > > > >> >     at org.apache.hadoop.ipc.Client.call(Client.java:1066)
>>> > > > >> >     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>> > > > >> >     at $Proxy2.create(Unknown Source)
>>> > > > >> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> > > > >> >     at java.lang.reflect.Method.invoke(Method.java:616)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>> > > > >> >     at $Proxy2.create(Unknown Source)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:3245)
>>> > > > >> >     at
>>> org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:713)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:182)
>>> > > > >> >     at
>>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
>>> > > > >> >     at
>>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
>>> > > > >> >     at
>>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
>>> > > > >> >     at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:229)
>>> > > > >> >     at
>>> > > > >> >
>>> > > >
>>> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1195)
>>> > > > >> >     at
>>> > > > >> >
>>> > > >
>>> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1171)
>>> > > > >> >     at
>>> > > > >> >
>>> > > >
>>> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1143)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> >
>>> org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:349)
>>> > > > >> >     at
>>> > > > org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:294)
>>> > > > >> >     at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:218)
>>> > > > >> >     at
>>> > org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:225)
>>> > > > >> >     at
>>> > > org.apache.hama.examples.PiEstimator.main(PiEstimator.java:139)
>>> > > > >> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> > > > >> >     at java.lang.reflect.Method.invoke(Method.java:616)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>>> > > > >> >     at
>>> > > > >>
>>> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>>> > > > >> >     at
>>> > > > >>
>>> org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:39)
>>> > > > >> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> > > > >> >     at
>>> > > > >> >
>>> > > > >>
>>> > > >
>>> > >
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> > > > >> >     at java.lang.reflect.Method.invoke(Method.java:616)
>>> > > > >> >     at org.apache.hama.util.RunJar.main(RunJar.java:147)
>>> > > > >> >
>>> > > > >> > My hama verison is 0.5 and hadoop version is 0.20.205.
This
>>> error
>>> > > > seems
>>> > > > >> to
>>> > > > >> > comes from the
>>> > > "org.apache.hadoop.hdfs.protocol.ClientProtocol.create"
>>> > > > >> > method, this is a normal method. I am kind of confused...
>>> > > > >> >
>>> > > > >> > Thanks in advance.
>>> > > > >> >
>>> > > > >> > walker
>>> > > > >>
>>> > > > >>
>>> > > > >>
>>> > > > >> --
>>> > > > >> Best Regards, Edward J. Yoon
>>> > > > >> @eddieyoon
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message