hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ikapo...@csd.auth.gr
Subject Re: Problem with 0.6.3
Date Sat, 19 Oct 2013 13:20:32 GMT
Well I have a test app where it just reads the input, which is:

1 2 3 4 5
2 1 3
3 1 2 4
4 1 3
5 1

and I just send a message but it doesn't even start. The error is:

13/10/19 18:56:27 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.lang.NullPointerException
	at org.apache.hama.bsp.PartitioningRunner.bsp(PartitioningRunner.java:207)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at  
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:662)
13/10/19 18:56:27 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.util.concurrent.ExecutionException: java.lang.NullPointerException
	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
	at  
org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:313)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
	at org.apache.hama.bsp.PartitioningRunner.bsp(PartitioningRunner.java:207)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at  
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	... 1 more
13/10/19 18:56:30 INFO bsp.BSPJobClient: Current supersteps number: 1
13/10/19 18:56:30 INFO bsp.BSPJobClient: Job failed.
13/10/19 18:56:30 ERROR bsp.BSPJobClient: Error partitioning the input path.
Exception in thread "main" java.io.IOException: Runtime partition  
failed for the job.
	at org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465)
	at org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333)
	at org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
	at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:228)
	at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
	at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:235)
	at hamatest.HamaTest.main(HamaTest.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at  
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at  
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hama.util.RunJar.main(RunJar.java:146)


What is this NullPointerException? I've seen it when NumBspTasks  
wasn't specified, but I have job.setNumBspTasks(1) and still happens.
I am also running in local mode and the only thing specified in  
hama-site.xml is the hdfs.

Another issue with the same jar.
When I run it again on local mode and nothing is mentioned in  
hama-site.xml, the output is this:

13/10/19 19:12:48 WARN util.NativeCodeLoader: Unable to load  
native-hadoop library for your platform... using builtin-java classes  
where applicable
13/10/19 19:12:48 INFO bsp.FileInputFormat: Total input paths to process : 1
13/10/19 19:12:48 WARN snappy.LoadSnappy: Snappy native library not loaded
13/10/19 19:12:49 INFO bsp.FileInputFormat: Total input paths to process : 1
13/10/19 19:12:49 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
13/10/19 19:12:49 INFO bsp.LocalBSPRunner: Setting up a new barrier  
for 1 tasks!
13/10/19 19:12:52 INFO bsp.BSPJobClient: Current supersteps number: 1
13/10/19 19:12:52 INFO bsp.BSPJobClient: The total number of supersteps: 1
13/10/19 19:12:52 INFO bsp.BSPJobClient: Counters: 6
13/10/19 19:12:52 INFO bsp.BSPJobClient:    
org.apache.hama.bsp.JobInProgress$JobCounter
13/10/19 19:12:52 INFO bsp.BSPJobClient:     SUPERSTEPS=1
13/10/19 19:12:52 INFO bsp.BSPJobClient:     LAUNCHED_TASKS=1
13/10/19 19:12:52 INFO bsp.BSPJobClient:    
org.apache.hama.bsp.BSPPeerImpl$PeerCounter
13/10/19 19:12:52 INFO bsp.BSPJobClient:     SUPERSTEP_SUM=2
13/10/19 19:12:52 INFO bsp.BSPJobClient:     IO_BYTES_READ=35
13/10/19 19:12:52 INFO bsp.BSPJobClient:     TIME_IN_SYNC_MS=0
13/10/19 19:12:52 INFO bsp.BSPJobClient:     TASK_INPUT_RECORDS=6
Exception in thread "main"  
org.apache.hadoop.mapred.InvalidInputException: Input path does not  
exist: /tmp/hama-parts/job_localrunner_0001
	at org.apache.hama.bsp.FileInputFormat.listStatus(FileInputFormat.java:163)
	at  
org.apache.hama.bsp.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:45)
	at org.apache.hama.bsp.FileInputFormat.getSplits(FileInputFormat.java:180)
	at org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:337)
	at org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
	at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:228)
	at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
	at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:235)
	at hamatest.HamaTest.main(HamaTest.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at  
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at  
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hama.util.RunJar.main(RunJar.java:146)

I don't understand again why this is happening. The input path I have  
specified is not "/tmp/hama-parts/job_localrunner_0001". I didn't  
repsond immediately cause I wanted to search it more by myself but I  
give up.
Hope you can help.

Ilias



Quoting Anastasis Andronidis <andronat_asf@hotmail.com>:

> Hey,
>
> I suggest you try the following as a general debug approach:
>
> 1) Can you test your input alone with a simple job that is just  
> parsing it? You can be sure at this point that something is not  
> wrong inside you code. Like lib dependences with problems, etc.
> 2) Check your job config. You might have keys and values to your  
> classes that are not fine represented, so Hama can not parse it.
> 3) Try to run Hama in local mode, just to be sure it's not a network problem.
>
> If you still have problem, can you send a part of your input that  
> you just checked that fails? So we can search for what is wrong.
>
> Cheers,
> Anastasis
>
> On 17 Οκτ 2013, at 9:34 μ.μ., Ηλίας Καπουράνης <ikapoura@csd.auth.gr>
wrote:
>
>> Hey,
>>
>> I only have 1 job. I don't know why it starts 2 jobs.
>>
>>
>>
>> Στις 17/10/2013 9:08 μμ, ο/η Anastasis Andronidis έγραψε:
>>> Hi,
>>>
>>> this seams to be a problem with the input. If you say that your  
>>> first job produces nothing, then the second job is getting null  
>>> input and this is probably why you get this error. I think your  
>>> problem might be in your first job.
>>>
>>> Can you separate the jobs and investigate why this is happening?  
>>> Or can you create a custom input for your second job and run it  
>>> with a specific input to be sure that the problem is there?
>>>
>>> Cheers,
>>> Anastasis
>>>
>>> On 17 Οκτ 2013, at 6:44 μ.μ., ikapoura@csd.auth.gr wrote:
>>>
>>>> Hello there,
>>>>
>>>>  i just downloaded the version 0.6.3, did the configurations and  
>>>> wanted to run my programs. Previously I was using hama 0.6.0. The  
>>>> examples are working fine but when I give one jar of mine to be  
>>>> run, I have the following issue:
>>>>
>>>> - It starts 2 jobs and after the first is completed without doing  
>>>> anything, the second hangs and prints this:
>>>>
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO  
>>>> ipc.Server: Starting Socket Reader #1 for port 52820
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO  
>>>> ipc.Server: IPC Server Responder: starting
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO  
>>>> ipc.Server: IPC Server listener on 52820: starting
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO  
>>>> ipc.Server: IPC Server handler 0 on 52820: starting
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO  
>>>> message.HadoopMessageManagerImpl:  BSPPeer address:master  
>>>> port:52820
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO  
>>>> sync.ZKSyncClient: Initializing ZK Sync Client
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:36 INFO  
>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At  
>>>> master/192.168.1.2:52820
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 ERROR  
>>>> bsp.BSPTask: Error running bsp setup and bsp function.
>>>> attempt_201310172123_0009_000000_0: java.lang.NullPointerException
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:430)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:114)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO  
>>>> ipc.Server: Stopping server on 52820
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO  
>>>> ipc.Server: IPC Server handler 0 on 52820: exiting
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO  
>>>> ipc.Server: Stopping IPC Server listener on 52820
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 INFO  
>>>> ipc.Server: Stopping IPC Server Responder
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 ERROR  
>>>> bsp.BSPTask: Shutting down ping service.
>>>> attempt_201310172123_0009_000000_0: 13/10/17 21:37:37 FATAL  
>>>> bsp.GroomServer: Error running child
>>>> attempt_201310172123_0009_000000_0: java.lang.NullPointerException
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:430)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:114)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
>>>> attempt_201310172123_0009_000000_0: java.lang.NullPointerException
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.graph.GraphJobRunner.loadVertices(GraphJobRunner.java:430)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.graph.GraphJobRunner.setup(GraphJobRunner.java:114)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
>>>> attempt_201310172123_0009_000000_0: 	at  
>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
>>>> 13/10/17 21:38:44 INFO bsp.BSPJobClient: Job failed.
>>>>
>>>>
>>>> From what I see, it has to do something either with the setup and  
>>>> bsp functions or with the reader because there is  
>>>> NullPointerException at loadVertices.
>>>>
>>>> I don't think it is the first one cause I am using hama Graph.
>>>>
>>>> Any ideas?
>>>> Thanks
>>>>
>>>>
>>
>>




Mime
View raw message