mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stratos Dimopoulos <stratos.dimopou...@gmail.com>
Subject Re: CDH5.2.3 on mesos
Date Fri, 31 Oct 2014 21:29:23 GMT
High Availability you mean? Why not Zookeeper?

On Thu, Oct 30, 2014 at 7:07 PM, Ankur Chauhan <ankur@malloc64.com> wrote:

> Thanks this is great. I actually ended up using the vanilla Hadoop
> distribution and it worked just fine. I will try out your tutorial. Side
> question, is there a solution around making the jobtracker ha?
>
> -- ankur
>
> Sent from my iPhone
>
> On Oct 30, 2014, at 5:05 PM, Stratos Dimopoulos <
> stratos.dimopoulos@gmail.com> wrote:
>
> Hi Ankur,
>
> I recently went through the process of installing Hadoop on Mesos over
> cdh5.1.2
>
> I created a post here - shouldn't be much different for your version:
> http://strat0sphere.wordpress.com/2014/10/30/hadoop-on-mesos-installation-guide/
>
> You can also find an other post about configuring CDH5.1.2 specifically to
> use with Mesos: useful:
> http://strat0sphere.wordpress.com/2014/10/30/cloudera-hdfs-cdh5-installation-to-use-with-mesos/
>
> Have in mind that when using Mesos you don't need to start the jobtracker.
> Mesos will do this for you.
> You also mentioned that you are trying to start hadoop as ubuntu user.
> This is not the right thing to do. Either add root to cloudera's root list
> or  (recommended) use the root user (mapred? hdfs?) that your cloudera
> version considers as root - you have to check the documentation for this.
>
> Regarding the error you are seeing "Does not contain a valid host:port
> authority: local" - I've seen this error when my worker version was
> different than the jobtracker version (happened because I was using a
> hadoop-on-mesos tar file compiled with a different version than the one my
> cluster was using). To fix this you can do the obvious, which is making
> sure the installed version is the same with the one you ship to the
> executors through HDFS or you can hack this by adding the property hadoop.skip.worker.version.check
> to True - In the later case I wish you good luck... Neverhteless, I am not
> sure if this error can also appear in other cases.
>
> Hope this helps.
>
> Stratos
>
>
> On Tue, Oct 28, 2014 at 12:30 PM, Ankur Chauhan <ankur@malloc64.com>
> wrote:
>
>> Anyone else have something to add on this?
>> -- Ankur Chauhan
>>
>> On 28 Oct 2014, at 02:10, Ankur Chauhan <ankur@malloc64.com> wrote:
>>
>> Hi tom,
>>
>> I was basically following the readme. This gist has the list of commands
>> how i am setting up things
>> https://gist.github.com/ankurcha/a9504b0e423b1a40d756 so first of all if
>> possible if you could help me verify if my process of setting up core-site,
>> hdfs-site and mapred-site is correct. I was starting the node with
>>  $ /opt/hadoop/bin/hadoop jobtracker
>>
>> There are two errors that i was working through. It seems that hadoop
>> doesn't like running as root (which is good) but despite starting the
>> process as ubuntu i kept getting
>>
>> Does not contain a valid host:port authority: local
>>
>> -- Ankur
>>
>> On 28 Oct 2014, at 01:57, Tom Arnfeld <tom@duedil.com> wrote:
>>
>> Hi Ankur,
>>
>> There aren't any getting started resources other than the documention
>> there as far as I know. Could you share your hadoop configuration and
>> perhaps a description of the problems you're having?
>>
>> Tom.
>>
>>
>>
>> On Tue, Oct 28, 2014 at 8:53 AM, Ankur Chauhan <ankur@malloc64.com>
>> wrote:
>>
>>> H,
>>>
>>> I was trying to setup mesos/hadoop with the latest CDH version (MR1) and
>>> it seems like the instructions are sort of out of date and I also tried the
>>> suggestions in https://github.com/mesos/hadoop/issues/25 but after 4
>>> hours of flailing around I am still kind of stuck :-/
>>>
>>> It seems like the configuration/installation instructions aren't
>>> complete and I am just too new to hadoop to figure out what's missing or
>>> going wrong. Does anyone know of a good resource I can use to get going?
>>>
>>> -- Ankur
>>>
>>
>>
>>
>>
>

Mime
View raw message