Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 18986200B22 for ; Wed, 1 Jun 2016 12:40:01 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 16F66160A46; Wed, 1 Jun 2016 10:40:01 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 3BAAF160A41 for ; Wed, 1 Jun 2016 12:40:00 +0200 (CEST) Received: (qmail 87644 invoked by uid 500); 1 Jun 2016 10:39:59 -0000 Mailing-List: contact dev-help@hama.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hama.apache.org Delivered-To: mailing list dev@hama.apache.org Received: (qmail 87633 invoked by uid 99); 1 Jun 2016 10:39:59 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Jun 2016 10:39:59 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 4479A2C033A for ; Wed, 1 Jun 2016 10:39:59 +0000 (UTC) Date: Wed, 1 Jun 2016 10:39:59 +0000 (UTC) From: "Edward J. Yoon (JIRA)" To: dev@hama.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HAMA-992) Hama streaming MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 01 Jun 2016 10:40:01 -0000 [ https://issues.apache.org/jira/browse/HAMA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310101#comment-15310101 ] Edward J. Yoon commented on HAMA-992: ------------------------------------- Thomas, I'm fine either way but adding to hama repo would be good idea if we can put some efforts to increase python supports and attract python users. > Hama streaming > -------------- > > Key: HAMA-992 > URL: https://issues.apache.org/jira/browse/HAMA-992 > Project: Hama > Issue Type: Question > Components: bsp core, pipes > Affects Versions: 0.7.1 > Environment: RASPBIAN JESSIE > Full desktop image based on Debian Jessie > Reporter: Chaitanya > Labels: features, github-import, newbie > > Hello all, > I am trying to implement apache hama on Raspberry pi model 3 to establish a distributed computing platform for scientific computation. I am trying to run hama streaming over hadoop on a single namenode but I am facing a bit of a difficulty in streaming my python code. I have downloaded the hama streaming repository from :- > https://github.com/thomasjungblut/HamaStreaming > I ran the examples and also HelloWorldBSP.py on Hama and they work well. But as soon as I switch to running my python code, the job fails. > I am trying to run the code with the following command:- > hama pipes -streaming true -bspTasks 1 -interpreter python -output /tmp/pystream-out_2/ -program /tmp/PyStreaming/BSPRunner.py -programArgs python.py > Below is the log file for your reference. I hope you can find time to help me in this minor project:- > 16/06/01 14:48:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable > 16/06/01 14:48:49 INFO ipc.Server: Starting Socket Reader #1 for port 61001 > 16/06/01 14:48:49 INFO ipc.Server: IPC Server listener on 61001: starting > 16/06/01 14:48:49 INFO ipc.Server: IPC Server Responder: starting > 16/06/01 14:48:49 INFO ipc.Server: IPC Server handler 0 on 61001: starting > 16/06/01 14:48:49 INFO ipc.Server: IPC Server handler 1 on 61001: starting > 16/06/01 14:48:49 INFO ipc.Server: IPC Server handler 3 on 61001: starting > 16/06/01 14:48:49 INFO ipc.Server: IPC Server handler 2 on 61001: starting > 16/06/01 14:48:49 INFO ipc.Server: IPC Server handler 4 on 61001: starting > 16/06/01 14:48:49 INFO message.HamaMessageManagerImpl: BSPPeer address:localhost port:61001 > 16/06/01 14:48:51 INFO Configuration.deprecation: mapred.cache.localFiles is deprecated. Instead, use mapreduce.job.cache.local.files > 16/06/01 14:48:51 INFO sync.ZKSyncClient: Initializing ZK Sync Client > 16/06/01 14:48:51 INFO sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At localhost/127.0.0.1:61001 > java.lang.NumberFormatException: For input string: "Traceback (most recent call last):" > at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) > at java.lang.Integer.parseInt(Integer.java:580) > at java.lang.Integer.parseInt(Integer.java:615) > at org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174) > at org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106) > 16/06/01 14:48:52 ERROR protocol.UplinkReader: java.lang.Exception: Bad command code: -2 > at org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:174) > java.util.concurrent.BrokenBarrierException > at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250) > at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362) > at org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223) > at org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293) > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43) > at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:170) > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:144) > at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255) > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException: java.lang.Exception: Bad command code: -2 > at org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:182) > Caused by: java.lang.Exception: Bad command code: -2 > at org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:174) > 16/06/01 14:48:52 ERROR bsp.BSPTask: Error running bsp setup and bsp function. > java.io.IOException: Stream closed > at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433) > at java.io.OutputStream.write(OutputStream.java:116) > at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141) > at java.io.DataOutputStream.flush(DataOutputStream.java:123) > at org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287) > at org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248) > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45) > at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:170) > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:144) > at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255) > 16/06/01 14:48:52 ERROR bsp.BSPTask: Error cleaning up after bsp executed. > java.io.IOException: Stream closed > at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433) > at java.io.OutputStream.write(OutputStream.java:116) > at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141) > at java.io.DataOutputStream.flush(DataOutputStream.java:123) > at org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287) > at org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271) > at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95) > at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:177) > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:144) > at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255) > 16/06/01 14:48:53 INFO ipc.Server: Stopping server on 61001 > 16/06/01 14:48:53 INFO ipc.Server: IPC Server handler 1 on 61001: exiting > 16/06/01 14:48:53 INFO ipc.Server: Stopping IPC Server listener on 61001 > 16/06/01 14:48:53 INFO ipc.Server: IPC Server handler 0 on 61001: exiting > 16/06/01 14:48:53 INFO ipc.Server: IPC Server handler 3 on 61001: exiting > 16/06/01 14:48:53 INFO ipc.Server: IPC Server handler 4 on 61001: exiting > 16/06/01 14:48:53 INFO ipc.Server: IPC Server handler 2 on 61001: exiting > 16/06/01 14:48:53 INFO ipc.Server: Stopping IPC Server Responder > 16/06/01 14:48:53 ERROR bsp.BSPTask: Shutting down ping service. > 16/06/01 14:48:53 FATAL bsp.GroomServer: Error running child > java.io.IOException: Stream closed > at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433) > at java.io.OutputStream.write(OutputStream.java:116) > at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141) > at java.io.DataOutputStream.flush(DataOutputStream.java:123) > at org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287) > at org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248) > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45) > at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:170) > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:144) > at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255) > java.io.IOException: Stream closed > at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433) > at java.io.OutputStream.write(OutputStream.java:116) > at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141) > at java.io.DataOutputStream.flush(DataOutputStream.java:123) > at org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287) > at org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248) > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45) > at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:170) > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:144) > at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255) -- This message was sent by Atlassian JIRA (v6.3.4#6332)