hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HAMA-978) NumberFormatException at StreamingUplinkReaderThread.readCommand
Date Tue, 01 Dec 2015 00:41:11 GMT

     [ https://issues.apache.org/jira/browse/HAMA-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Edward J. Yoon resolved HAMA-978.
---------------------------------
    Resolution: Fixed
      Assignee: Edward J. Yoon

This bug has fixed by https://github.com/apache/hama/commit/c095c7da2e529256579429da9bb2d534a96da873

> NumberFormatException at StreamingUplinkReaderThread.readCommand
> ----------------------------------------------------------------
>
>                 Key: HAMA-978
>                 URL: https://issues.apache.org/jira/browse/HAMA-978
>             Project: Hama
>          Issue Type: Bug
>          Components: pipes
>    Affects Versions: 0.7.0
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 0.7.1
>
>
> {code}
> Hi to all,
> this is my first mail to this mailing list so please have patience if I
> make some bad choice in the format.
> I have a kubuntu-14.04 on an old Intel Core 2 Duo processor T7500 with 2 GB
> of RAM.
> I have properly installed Hadoop-2.7.1, Sun Java JDK 1.8.0_60 e Hama-0.7.0
> as you can see from the following lines:
> >
> > > $ hadoop version
> > > Hadoop 2.7.1
> > > Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
> 15ecc87ccf4a0228f35af08fc56de536e6ce657a
> > > Compiled by jenkins on 2015-06-29T06:04Z
> > > Compiled with protoc 2.5.0
> > > From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
> > > This command was run using
> /home/tora/Downloads/hadoop-2.7.1/share/hadoop/common/hadoop-common-2.7.1.jar
> > > $ java -version
> > > java version "1.8.0_60"
> > > Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> > > Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> > >
> I am able to properly run the basic Hadoop and Hama examples like the
> following:
> bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
> grep input output 'dfs[a-z.]+'
> bin/hama jar hama-examples-0.7.0.jar pi
> My problem is that I receive the following error message when I try to run
> the HelloWorld Hama Streaming example with this instruction:
> bin/hama pipes -streaming true -bspTasks 2 -interpreter python3.2
> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ -program
> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> The default python interpreters of my OS were python-2.7 and python-3.4;
> since I had problems with this example I also tried to install python-3.2
> with the following instructions but it didn't solved the problem:
> >
> > > sudo apt-get install software-properties-common
> > > sudo apt-add-repository ppa:fkrull/deadsnakes
> > > sudo apt-get update
> > > sudo apt-get install python3.2
> > >
> The installed python version is 3.2.6 as you can see from the following
> lines:
> >
> > > $ python3.2
> > > Python 3.2.6 (default, Oct 21 2014, 12:50:03)
> > > [GCC 4.8.2] on linux2
> > > Type "help", "copyright", "credits" or "license" for more information.
> > > >>>
> > >
> The error message is the following (I am working in local mode so I didn't
> run bin/start-bspd.sh):
> >
> >
> > >
> > > $ clear;bin/hama pipes -streaming true -bspTasks 2 -interpreter
> python3.2 -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> -program /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> > > 15/09/30 12:39:11 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> > > 15/09/30 12:39:11 INFO pipes.Submitter: Streaming enabled!
> > > 15/09/30 12:39:11 INFO Configuration.deprecation: fs.default.name is
> deprecated. Instead, use fs.defaultFS
> > > 15/09/30 12:39:11 INFO Configuration.deprecation: user.name is
> deprecated. Instead, use mapreduce.job.user.name
> > > 15/09/30 12:39:11 WARN conf.Configuration:
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
> > > 15/09/30 12:39:11 WARN conf.Configuration:
> org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream@4ddced80:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts;  Ignoring.
> > > 15/09/30 12:39:12 INFO Configuration.deprecation: user.name is
> deprecated. Instead, use mapreduce.job.user.name
> > > 15/09/30 12:39:12 INFO bsp.BSPJobClient: Running job:
> job_localrunner_0001
> > > 15/09/30 12:39:12 INFO Configuration.deprecation:
> mapred.cache.localFiles is deprecated. Instead, use
> mapreduce.job.cache.local.files
> > > 15/09/30 12:39:12 INFO bsp.LocalBSPRunner: Setting up a new barrier for
> 2 tasks!
> > > java.lang.NumberFormatException: For input string: "Traceback (most
> recent call last):"
> > >         at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > >         at java.lang.Integer.parseInt(Integer.java:580)
> > >         at java.lang.Integer.parseInt(Integer.java:615)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > > java.lang.NumberFormatException: For input string: "Traceback (most
> recent call last):"
> > >         at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > >         at java.lang.Integer.parseInt(Integer.java:580)
> > >         at java.lang.Integer.parseInt(Integer.java:615)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol$StreamingUplinkReaderThread.readCommand(StreamingProtocol.java:174)
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:106)
> > > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >
> > > 15/09/30 12:39:12 ERROR protocol.UplinkReader: java.lang.Exception: Bad
> command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >
> > > java.util.concurrent.BrokenBarrierException
> > >         at
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > >         at
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > >         at
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > >         at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >         at java.lang.Thread.run(Thread.java:745)
> > > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> java.lang.Exception: Bad command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > > Caused by: java.lang.Exception: Bad command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > > java.util.concurrent.BrokenBarrierException
> > >         at
> java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:250)
> > >         at
> java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.start(StreamingProtocol.java:223)
> > >         at
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:293)
> > >         at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:43)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >         at java.lang.Thread.run(Thread.java:745)
> > > Exception in thread "pipe-uplink-handler" java.lang.RuntimeException:
> java.lang.Exception: Bad command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:180)
> > > Caused by: java.lang.Exception: Bad command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
> > > java.io.IOException: Stream closed
> > >         at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >         at java.io.OutputStream.write(OutputStream.java:116)
> > >         at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >         at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >         at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >         at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.runSetup(StreamingProtocol.java:248)
> > >         at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:45)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >         at java.lang.Thread.run(Thread.java:745)
> > > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
> > > java.lang.InterruptedException: java.lang.Exception: Bad command code:
> -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >
> > >         at
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> > >         at
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> > >         at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >         at java.lang.Thread.run(Thread.java:745)
> > > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> execution.
> > > java.io.IOException: Stream closed
> > >         at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >         at java.io.OutputStream.write(OutputStream.java:116)
> > >         at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >         at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >         at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >         at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > >         at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >         at java.lang.Thread.run(Thread.java:745)
> > > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Error cleaning up after bsp
> execution.
> > > java.io.IOException: Stream closed
> > >         at
> java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:433)
> > >         at java.io.OutputStream.write(OutputStream.java:116)
> > >         at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> > >         at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> > >         at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
> > >         at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.writeLine(StreamingProtocol.java:287)
> > >         at
> org.apache.hama.pipes.protocol.StreamingProtocol.runCleanup(StreamingProtocol.java:271)
> > >         at org.apache.hama.pipes.PipesBSP.cleanup(PipesBSP.java:95)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:262)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >         at java.lang.Thread.run(Thread.java:745)
> > > 15/09/30 12:39:12 ERROR bsp.LocalBSPRunner: Exception during BSP
> execution!
> > > java.util.concurrent.ExecutionException:
> java.lang.InterruptedException: java.lang.Exception: Bad command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >
> > >         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> > >         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
> > >         at java.lang.Thread.run(Thread.java:745)
> > > Caused by: java.lang.InterruptedException: java.lang.Exception: Bad
> command code: -2
> > >         at
> org.apache.hama.pipes.protocol.UplinkReader.run(UplinkReader.java:172)
> > >
> > >         at
> org.apache.hama.pipes.protocol.BinaryProtocol.waitForFinish(BinaryProtocol.java:335)
> > >         at
> org.apache.hama.pipes.PipesApplication.waitForFinish(PipesApplication.java:329)
> > >         at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:47)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> > >         at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > >         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > >         ... 1 more
> > > 15/09/30 12:39:13 INFO bsp.BSPJobClient: Job failed.
> > >
> I tried to searching for these error messages with google but I didn't find
> any solution.
> Does anyone of you have any idea of how can I solve this problem?
> Any help is appreciated :)
> Thank you
> Best
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message