hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stratos Dimopoulos <stratos.dimopou...@gmail.com>
Subject Re: Using hama with Mesos 0.20 and CDH5.1.2
Date Sat, 25 Oct 2014 02:20:23 GMT
And when you added the list's user email you had a typo which I copy pasted
and the email never reached the list - So I am just forwarding again to the
correct address this time! :-)

On Thu, Oct 23, 2014 at 5:36 PM, Stratos Dimopoulos <
stratos.dimopoulos@gmail.com> wrote:

> Yes Jeff correct - this is the way I compile. Here is the stack trace
> close to the error on the executor - I don't think the log file on the BSP
> master would help you at all but let me know if you need to send you
> anything else:
>
> I1022 17:25:31.828336 22440 exec.cpp:132] Version: 0.20.0
> I1022 17:25:31.838382 22454 exec.cpp:206] Executor registered on slave
> 20141019-011259-1257767434-5050-4023-1
> 14/10/22 17:25:31 INFO bsp.MesosExecutor: Executor registered with the
> slave
> 14/10/22 17:25:31 INFO bsp.MesosExecutor: Launching task : Task_0
> 14/10/22 17:25:32 INFO bsp.GroomServer: groom start
> 14/10/22 17:25:32 INFO bsp.GroomServer:
> /mnt/mesos-work-dir/slaves/20141019-011259-1257767434-5050-4023-1/frameworks/20141019-011259-1257767434-5050-4023-0031/executors/executor_Task_0/runs/d69f21af-9180-4c0c-9263-82038ae81937/hama-0.7.0-SNAPSHOT/bsp/local
> 14/10/22 17:25:32 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/10/22 17:25:33 ERROR bsp.MesosExecutor: Caught exception, committing
> suicide.
> java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem:
> Provider org.apache.hadoop.fs.LocalFileSystem not found
>         at java.util.ServiceLoader.fail(ServiceLoader.java:231)
>         at java.util.ServiceLoader.access$300(ServiceLoader.java:181)
>         at
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:365)
>         at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
>         at
> org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2364)
>         at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2375)
>         at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2392)
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
>         at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2431)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
>         at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:339)
>         at
> org.apache.hama.bsp.GroomServer.deleteLocalFiles(GroomServer.java:483)
>         at org.apache.hama.bsp.GroomServer.initialize(GroomServer.java:321)
>         at org.apache.hama.bsp.GroomServer.run(GroomServer.java:860)
>         at org.apache.hama.bsp.MesosExecutor$1.run(MesosExecutor.java:92)
>
> thanks,
> Stratos
>
> On Thu, Oct 23, 2014 at 2:51 PM, Jeff Fenchel <jfenc91@gmail.com> wrote:
>
>> Fyi... Sorry I forgot to add your email to list. Please add
>> user@hama.apach.org to your reply.
>>
>> Thanks,
>> Jeff
>>
>> ---------- Forwarded message ----------
>> From: *Jeff Fenchel* <jfenc91@gmail.com>
>> Date: Thursday, October 23, 2014
>> Subject: Using hama with Mesos 0.20 and CDH5.1.2
>> To: user@hama.apache.org
>>
>>
>> Hey Stratos,
>>
>> Thanks for giving Hama on Mesos a try. I do not think it has ever been
>> tested with hadoop 2, but I see no reason why it shouldn't work. So, I am
>>
>> guessing you compiled with something like the following?
>>
>> mvn clean install -Phadoop2 -Dhadoop.version=2.3.0-cdh5.1.2
>> -Dmesos.version=0.20.0 -DskipTests
>>
>> If so, can you please send us the complete stack trace? It does look
>> like this might be a bug in hama.
>>
>> Thanks,
>> -Jeff
>>
>>
>>
>>
>> On Thu, Oct 23, 2014 at 1:13 AM, Stratos Dimopoulos
>> <stratos.dimopoulos@gmail.com> wrote:
>> > Hi,
>> >
>> > I cloned Hama from Git and installed in a Eucalyptus cloud with Mesos
>> 0.20
>> > and CDH5.1.2 HDFS. It is also configured with zookeeper but my
>> zookeeper at
>> > the moment just runs in one master. I followed the instructions here:
>> > http://wiki.apache.org/hama/GettingStartedMesos - that are for an older
>> > version and additionally to that I added some more properties to
>> overcome
>> > some runtime problems and until Mesos was able to recognize Hama as a
>> > framework. Overall the properties I have set are the following:
>> >
>> > bsp.master.address
>> > bsp.master.port
>> > bsp.master.TaskWorkerManager.class
>> > fs.default.name (I don't think it needs to be here but in an effort to
>> fix
>> > the issues I had)
>> > hama.mesos.executor.uri
>> > bsp.tasks.maximum.total
>> > hama.mesos.master
>> > bsp.child.java.opts
>> > bsp.system.dir
>> > bsp.local.dir
>> > bsp.tmp.dir
>> > bsp.disk.queue.dir
>> > hama.zookeeper.quorum
>> > hama.zookeeper.property.clientPort
>> >
>> > When I run the example the executor is running but the task fails. The
>> > error output on the executors log is: ERROR bsp.MesosExecutor: Caught
>> > exception, committing suicide.
>> > java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem:
>> > Provider org.apache.hadoop.fs.LocalFileSystem not found
>> >
>> > I did some research on this error message but I cannot find something
>> that
>> > really helps. It seems to me as a configuration issue but I don't know
>> how
>> > to solve it.
>> > Also note that I have verified that my HDFS works properly. I am
>> actually
>> > already running Spark and Hadoop over Mesos without problems.
>> >
>> > I hope somebody can help!
>> >
>> > thanks,
>> > Stratos
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message