hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Illecker (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-805) Problem initializing pipes in HamaStreaming
Date Tue, 24 Sep 2013 09:41:06 GMT

    [ https://issues.apache.org/jira/browse/HAMA-805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776156#comment-13776156
] 

Martin Illecker commented on HAMA-805:
--------------------------------------

I don't think it's a HDFS's distcache problem.

{code}
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG hama.pipes.logging: false
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFilesCount: 9
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BSPRunner.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BSPRunner.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BSP.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BSPPeer.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BSPRunner.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BinaryProtocol.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BspJobConfiguration.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/HelloWorldBSP.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG LocalCacheFiles: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/KMeansBSP.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG: executable: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BSPRunner.py
13/09/24 11:30:44 DEBUG pipes.PipesApplication: File: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/tasklogs/job_201309241128_0001
created!
13/09/24 11:30:44 DEBUG pipes.PipesApplication: STDOUT: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/tasklogs/job_201309241128_0001/attempt_201309241128_0001_000000_0.log
13/09/24 11:30:44 DEBUG pipes.PipesApplication: STDERR: /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/tasklogs/job_201309241128_0001/attempt_201309241128_0001_000000_0.err
13/09/24 11:30:44 DEBUG pipes.PipesApplication: DEBUG: cmd: [bash, -c, 'python3.2' '/tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/BSPRunner.py'
'HelloWorldBSP'  2>&1 | tee /tmp/hadoop-martin/bsp/local/groomServer/attempt_201309241128_0001_000000_0/work/tasklogs/job_201309241128_0001/attempt_201309241128_0001_000000_0.log]
{code}

Hama Streaming was developed by Thomas Jungblut nearly a year ago and I believe the Streaming
API is not up to date.

{code}
13/09/24 11:30:44 ERROR protocol.UplinkReader: java.io.IOException: Bad command code: 98
	at org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:173)
{code}

I will have a look.

                
> Problem initializing pipes in HamaStreaming
> -------------------------------------------
>
>                 Key: HAMA-805
>                 URL: https://issues.apache.org/jira/browse/HAMA-805
>             Project: Hama
>          Issue Type: Bug
>         Environment: python 3.2, hadoop2
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 0.6.3
>
>
> {code}
> Hello all,
> I try to use Hama Streaming.
> I have successfully installed Hama (the Pi example works).
> I follow this tutorial:
> http://wiki.apache.org/hama/HamaStreaming
> When I try to run the distributed HelloWorld in the local
> configuration, I get the following error:
> $ bin/hama pipes -streaming true -bspTasks 3 -interpreter python3.2
> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> 13/09/23 18:03:50 INFO pipes.Submitter: Streaming enabled!
> 13/09/23 18:03:50 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 13/09/23 18:03:50 WARN bsp.BSPJobClient: No job jar file set.  User
> classes may not be found. See BSPJob#setJar(String) or check Your jar
> file.
> 13/09/23 18:03:50 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
> 13/09/23 18:03:50 INFO bsp.LocalBSPRunner: Setting up a new barrier for 3 tasks!
> 13/09/23 18:03:50 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
> java.lang.NullPointerException
>    at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44)
>    at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>    at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>    at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>    at java.lang.Thread.run(Thread.java:662)
> [output cropped]
> When I turn to the pseudo-distributed mode, job fails too (after a
> minute of execution):
> 13/09/23 18:46:34 INFO pipes.Submitter: Streaming enabled!
> 13/09/23 18:46:34 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 13/09/23 18:46:34 WARN bsp.BSPJobClient: No job jar file set.  User
> classes may not be found. See BSPJob#setJar(String) or check Your jar
> file.
> 13/09/23 18:46:34 INFO bsp.BSPJobClient: Running job: job_201309231846_0001
> 13/09/23 18:47:40 INFO bsp.BSPJobClient: Job failed.
> Task log contains errors:
> 13/09/23 18:46:37 INFO ipc.Server: Starting Socket Reader #1 for port 43475
> 13/09/23 18:46:37 INFO ipc.Server: IPC Server Responder: starting
> 13/09/23 18:46:37 INFO ipc.Server: IPC Server listener on 43475: starting
> 13/09/23 18:46:37 INFO message.HadoopMessageManagerImpl:  BSPPeer
> address:localhost.localdomain port:43475
> 13/09/23 18:46:37 INFO ipc.Server: IPC Server handler 0 on 43475: starting
> 13/09/23 18:46:37 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 13/09/23 18:46:37 INFO sync.ZKSyncClient: Initializing ZK Sync Client
> 13/09/23 18:46:37 INFO sync.ZooKeeperSyncClientImpl: Start connecting
> to Zookeeper! At localhost.localdomain/127.0.0.1:43475
> 13/09/23 18:46:37 ERROR bsp.BSPTask: Error running bsp setup and bsp function.
> java.lang.NullPointerException
>    at java.io.File.<init>(File.java:222)
>    at org.apache.hama.pipes.PipesApplication.setupCommand(PipesApplication.java:130)
>    at org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:257)
>    at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44)
>    at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
>    at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
>    at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
> [output cropped]
> I use the latest trunk version of Hama, Python 3.2.5 and Hadoop 2.0.0-cdh4.1.1.
> Please help me to figure out the problem.
> Thanks in advance,
> Roman
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message