hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leonidas Fegaras <fega...@cse.uta.edu>
Subject Problems running Hama v0.7 on Yarn
Date Wed, 15 Jul 2015 07:12:03 GMT
Hi,
I am extending MRQL to support Hama v0.7 on Yarn (see 
https://issues.apache.org/jira/browse/MRQL-75 ).
Currently, MRQL on Hama works fine on Mesos but I have problems running 
it on Yarn.

1) Without using the PartitioningRunner, the Yarn Application master 
crashes.
     It seems that the reason is that I have 1 input block (1 split) but 
I use 4 tasks.
     This may be caused by my input format.
     But the Application master shouldn't crash; it should have used 1 
task instead.
     The log is attached below.

2) If I use the PartitioningRunner using:
job.setPartitioner(org.apache.hama.bsp.HashPartitioner.class);
         job.setNumBspTask(4);
         job.set("bsp.min.split.size","102");
    it fails because it expects a Long key. Here is the log:

15/07/15 09:31:40 INFO bsp.BSPJobClient: Running job: job_localrunner_0001
15/07/15 09:31:42 INFO bsp.LocalBSPRunner: Setting up a new barrier for 
4 tasks!
15/07/15 09:31:42 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
java.io.IOException: wrong key class: org.apache.mrql.MRContainer is not 
class org.apache.hadoop.io.LongWritable
     at 
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1306)
     at 
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1298)
     at 
org.apache.hama.bsp.SequenceFileRecordWriter.write(SequenceFileRecordWriter.java:47)
     at 
org.apache.hama.bsp.SequenceFileRecordWriter.write(SequenceFileRecordWriter.java:31)
     at org.apache.hama.bsp.BSPPeerImpl$1.collect(BSPPeerImpl.java:335)
     at org.apache.hama.bsp.BSPPeerImpl.write(BSPPeerImpl.java:628)
     at 
org.apache.hama.bsp.PartitioningRunner.bsp(PartitioningRunner.java:156)

Thanks,
Leonidas Fegaras


Mime
View raw message