hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shapovalov <shapova...@graphics.cs.msu.su>
Subject Re: Problem initializing pipes in HamaStreaming
Date Thu, 26 Sep 2013 17:20:27 GMT
Hi Edward,

Could you please be more specific? (Sorry, I am new to this stuff)

I run Hama in local mode. The logs/ directory is empty, and I did not
find any logs in HDFS as well.

And where can I find the Hadoop configuration?

Thank you,
Roman

On Thu, Sep 26, 2013 at 12:05 PM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Hi,
>
> That's strange. Can you attach your namenode logs and hadoop configurations?
>
> On Thu, Sep 26, 2013 at 11:03 PM, Roman Shapovalov
> <shapovalov@graphics.cs.msu.su> wrote:
>> Hi again,
>>
>> I have updated both Hama (from the trunk) and Streaming (from Martin's
>> github), and checked that patches have been applied, but I keep
>> getting the same error (full log for local configuration is attached).
>>
>> Another thing may be relevant: I keep the default Hadoop libraries in
>> lib/. If I replace them as the tutorial says, some classes cannot be
>> found even if  I run pure Hama (which works perfectly with default
>> libs). I don't know if it is important.
>>
>> Thanks,
>> Roman
>>
>> On Tue, Sep 24, 2013 at 9:22 AM, Martin Illecker <millecker@apache.org> wrote:
>>> Hi Roman,
>>>
>>> sorry for inconvenience!
>>> The problem has been reported [1] and will be fixed shortly to the trunk.
>>>
>>> [1] https://issues.apache.org/jira/browse/HAMA-805
>>>
>>>
>>> 2013/9/23 Edward J. Yoon <edwardyoon@apache.org>
>>>
>>>> This looks like a bug of DistCacheUtils.
>>>>
>>>> Thanks for your report. I'll look at it tomorrow.
>>>>
>>>> On Mon, Sep 23, 2013 at 11:52 PM, Roman Shapovalov
>>>> <shapovalov@graphics.cs.msu.su> wrote:
>>>> > Hello all,
>>>> >
>>>> > I try to use Hama Streaming.
>>>> > I have successfully installed Hama (the Pi example works).
>>>> > I follow this tutorial:
>>>> > http://wiki.apache.org/hama/HamaStreaming
>>>> >
>>>> > When I try to run the distributed HelloWorld in the local
>>>> > configuration, I get the following error:
>>>> >
>>>> > $ bin/hama pipes -streaming true -bspTasks 3 -interpreter python3.2
>>>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
>>>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
>>>> >
>>>> > 13/09/23 18:03:50 INFO pipes.Submitter: Streaming enabled!
>>>> > 13/09/23 18:03:50 WARN util.NativeCodeLoader: Unable to load
>>>> > native-hadoop library for your platform... using builtin-java classes
>>>> > where applicable
>>>> > 13/09/23 18:03:50 WARN bsp.BSPJobClient: No job jar file set.  User
>>>> > classes may not be found. See BSPJob#setJar(String) or check Your jar
>>>> > file.
>>>> > 13/09/23 18:03:50 INFO bsp.BSPJobClient: Running job:
>>>> job_localrunner_0001
>>>> > 13/09/23 18:03:50 INFO bsp.LocalBSPRunner: Setting up a new barrier
for
>>>> 3 tasks!
>>>> > 13/09/23 18:03:50 ERROR bsp.LocalBSPRunner: Exception during BSP
>>>> execution!
>>>> > java.lang.NullPointerException
>>>> >     at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44)
>>>> >     at
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
>>>> >     at
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>>>> >     at
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
>>>> >     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> >     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> >     at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> >     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> >     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> >     at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> >     at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> >     at java.lang.Thread.run(Thread.java:662)
>>>> > [output cropped]
>>>> >
>>>> > When I turn to the pseudo-distributed mode, job fails too (after a
>>>> > minute of execution):
>>>> >
>>>> > 13/09/23 18:46:34 INFO pipes.Submitter: Streaming enabled!
>>>> > 13/09/23 18:46:34 WARN util.NativeCodeLoader: Unable to load
>>>> > native-hadoop library for your platform... using builtin-java classes
>>>> > where applicable
>>>> > 13/09/23 18:46:34 WARN bsp.BSPJobClient: No job jar file set.  User
>>>> > classes may not be found. See BSPJob#setJar(String) or check Your jar
>>>> > file.
>>>> > 13/09/23 18:46:34 INFO bsp.BSPJobClient: Running job:
>>>> job_201309231846_0001
>>>> > 13/09/23 18:47:40 INFO bsp.BSPJobClient: Job failed.
>>>> >
>>>> > Task log contains errors:
>>>> >
>>>> > 13/09/23 18:46:37 INFO ipc.Server: Starting Socket Reader #1 for port
>>>> 43475
>>>> > 13/09/23 18:46:37 INFO ipc.Server: IPC Server Responder: starting
>>>> > 13/09/23 18:46:37 INFO ipc.Server: IPC Server listener on 43475: starting
>>>> > 13/09/23 18:46:37 INFO message.HadoopMessageManagerImpl:  BSPPeer
>>>> > address:localhost.localdomain port:43475
>>>> > 13/09/23 18:46:37 INFO ipc.Server: IPC Server handler 0 on 43475:
>>>> starting
>>>> > 13/09/23 18:46:37 WARN util.NativeCodeLoader: Unable to load
>>>> > native-hadoop library for your platform... using builtin-java classes
>>>> > where applicable
>>>> > 13/09/23 18:46:37 INFO sync.ZKSyncClient: Initializing ZK Sync Client
>>>> > 13/09/23 18:46:37 INFO sync.ZooKeeperSyncClientImpl: Start connecting
>>>> > to Zookeeper! At localhost.localdomain/127.0.0.1:43475
>>>> > 13/09/23 18:46:37 ERROR bsp.BSPTask: Error running bsp setup and bsp
>>>> function.
>>>> > java.lang.NullPointerException
>>>> >     at java.io.File.<init>(File.java:222)
>>>> >     at
>>>> org.apache.hama.pipes.PipesApplication.setupCommand(PipesApplication.java:130)
>>>> >     at
>>>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:257)
>>>> >     at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44)
>>>> >     at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
>>>> >     at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
>>>> >     at
>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
>>>> > [output cropped]
>>>> >
>>>> > I use the latest trunk version of Hama, Python 3.2.5 and Hadoop
>>>> 2.0.0-cdh4.1.1.
>>>> >
>>>> > Please help me to figure out the problem.
>>>> >
>>>> > Thanks in advance,
>>>> > Roman
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon
>>>> @eddieyoon
>>>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon

Mime
View raw message