hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Fenchel <jfen...@gmail.com>
Subject Re: Using hama with Mesos 0.20 and CDH5.1.2
Date Thu, 23 Oct 2014 09:13:41 GMT
Hey Stratos,

Thanks for giving Hama on Mesos a try. I do not think it has ever been
tested with hadoop 2, but I see no reason why it shouldn't work. So, I
guessing you compiled with something like the following?

mvn clean install -Phadoop2 -Dhadoop.version=2.3.0-cdh5.1.2
-Dmesos.version=0.20.0 -DskipTests

If so, can you please send us the complete stack trace? It does look
like this might be a bug in hama.

Thanks,
-Jeff




On Thu, Oct 23, 2014 at 1:13 AM, Stratos Dimopoulos
<stratos.dimopoulos@gmail.com> wrote:
> Hi,
>
> I cloned Hama from Git and installed in a Eucalyptus cloud with Mesos 0.20
> and CDH5.1.2 HDFS. It is also configured with zookeeper but my zookeeper at
> the moment just runs in one master. I followed the instructions here:
> http://wiki.apache.org/hama/GettingStartedMesos - that are for an older
> version and additionally to that I added some more properties to overcome
> some runtime problems and until Mesos was able to recognize Hama as a
> framework. Overall the properties I have set are the following:
>
> bsp.master.address
> bsp.master.port
> bsp.master.TaskWorkerManager.class
> fs.default.name (I don't think it needs to be here but in an effort to fix
> the issues I had)
> hama.mesos.executor.uri
> bsp.tasks.maximum.total
> hama.mesos.master
> bsp.child.java.opts
> bsp.system.dir
> bsp.local.dir
> bsp.tmp.dir
> bsp.disk.queue.dir
> hama.zookeeper.quorum
> hama.zookeeper.property.clientPort
>
> When I run the example the executor is running but the task fails. The
> error output on the executors log is: ERROR bsp.MesosExecutor: Caught
> exception, committing suicide.
> java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem:
> Provider org.apache.hadoop.fs.LocalFileSystem not found
>
> I did some research on this error message but I cannot find something that
> really helps. It seems to me as a configuration issue but I don't know how
> to solve it.
> Also note that I have verified that my HDFS works properly. I am actually
> already running Spark and Hadoop over Mesos without problems.
>
> I hope somebody can help!
>
> thanks,
> Stratos

Mime
View raw message