Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of daemeonr@gmail.com designates
 209.85.223.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAKKt98RqHecq1vRdS7P+MYFsxUym_iuUpNH2NHtEnoiqCB+c4A@mail.gmail.com>
References: 
 <CAGCH7McNMD=F7rTbiRJqWrDvNdVSfGw10u7HVAjGRm1YgzKaEg@mail.gmail.com>
 <0a49e536b65de41e39692a05333e8254@mail.eagleeyet.net>
 <CAGCH7McfP+qDbF9KZh+=Vf=NRxUsD+6Qcnx32au35khBmEhOww@mail.gmail.com>
 <285b89b414575517a0181dae0f6dfd87@mail.eagleeyet.net>
 <CAKKt98QCrfvNqcM+DVWWFZciyE3GM-KmXNDwt2ypLVLrAa2KWQ@mail.gmail.com>
 <9cdd28c6b02494f8ae91cd9abc50d276@mail.eagleeyet.net>
 <CAKKt98RqHecq1vRdS7P+MYFsxUym_iuUpNH2NHtEnoiqCB+c4A@mail.gmail.com>
From: daemeon reiydelle <daemeonr@gmail.com>
Date: Thu, 5 Mar 2015 22:43:51 -0800
Message-ID: 
 <CAOUOv0F6=PwteZhn+thv3HpTVtyOntVFqGEQzwTjC1rM8khZew@mail.gmail.com>
Subject: Re: t2.micro on AWS; Is it enough for setting up Hadoop cluster ?
To: user@hadoop.apache.org
Content-Type: text/plain; charset=UTF-8

Do a reverse lookup and use the name you find. There are a few areas
of Hadoopo that require reverse name lookup, but in general just
create relevant entries (shared across the cluster, e.g. via Ansible
if more than just a few nodes) in /etc/hosts.

Not hard.


On Thu, Mar 5, 2015 at 6:35 PM, Alexander Pivovarov
<apivovarov@gmail.com> wrote:
> I found the following solution to this problem
>
> I registered 2 subdomains  (public and local) for each computer on
> https://freedns.afraid.org/subdomain/
> e.g.
> myhadoop-nn.crabdance.com
> myhadoop-nn-local.crabdance.com
>
> then I added cron job which sends http requests to update public and local
> ip on freedns server
> hint: public ip is detected automatically
> ip address for local name can be set using request parameter
> &address=10.x.x.x   (don't forget to escape &)
>
> as a result my nn computer has 2 DNS names with currently assigned ip
> addresses , e.g.
> myhadoop-nn.crabdance.com  54.203.181.177
> myhadoop-nn-local.crabdance.com   10.220.149.103
>
> in hadoop configuration I can use local machine names
> to access my cluster outside of AWS I can use public names
>
> Just curious if AWS provides easier way to name EC2 computers?
>
> On Thu, Mar 5, 2015 at 5:19 PM, Jonathan Aquilina <jaquilina@eagleeyet.net>
> wrote:
>>
>> I dont know how you would do that to be honest. With EMR you have
>> destinctions master core and task nodes. If you need to change configuration
>> you just ssh into the EMR master node.
>>
>>
>>
>> ---
>> Regards,
>> Jonathan Aquilina
>> Founder Eagle Eye T
>>
>> On 2015-03-06 02:11, Alexander Pivovarov wrote:
>>
>> What is the easiest way to assign names to aws ec2 computers?
>> I guess computer need static hostname and dns name before it can be used
>> in hadoop cluster.
>>
>> On Mar 5, 2015 4:36 PM, "Jonathan Aquilina" <jaquilina@eagleeyet.net>
>> wrote:
>>>
>>> When I started with EMR it was alot of testing and trial and error. HUE
>>> is already supported as something that can be installed from the AWS
>>> console. What I need to know is if you need this cluster on all the time or
>>> this is goign ot be what amazon call a transient cluster. Meaning you fire
>>> it up run the job and tear it back down.
>>>
>>>
>>>
>>> ---
>>> Regards,
>>> Jonathan Aquilina
>>> Founder Eagle Eye T
>>>
>>> On 2015-03-06 01:10, Krish Donald wrote:
>>>
>>> Thanks Jonathan,
>>>
>>> I will try to explore EMR option also.
>>> Can you please let me know the configuration which you have used it?
>>> Can you please recommend for me also?
>>> I would like to setup Hadoop cluster using cloudera manager and then
>>> would like to do below things:
>>>
>>> setup kerberos
>>> setup federation
>>> setup monitoring
>>> setup hadr
>>> backup and recovery
>>> authorization using sentry
>>> backup and recovery of individual componenets
>>> performamce tuning
>>> upgrade of cdh
>>> upgrade of CM
>>> Hue User Administration
>>> Spark
>>> Solr
>>>
>>>
>>> Thanks
>>> Krish
>>>
>>>
>>> On Thu, Mar 5, 2015 at 3:57 PM, Jonathan Aquilina
>>> <jaquilina@eagleeyet.net> wrote:
>>>>
>>>> krish EMR wont cost you much with all the testing and data we ran
>>>> through the test systems as well as the large amont of data when everythign
>>>> was read we paid about 15.00 USD. I honestly do not think that the specs
>>>> there would be enough as java can be pretty ram hungry.
>>>>
>>>>
>>>>
>>>> ---
>>>> Regards,
>>>> Jonathan Aquilina
>>>> Founder Eagle Eye T
>>>>
>>>> On 2015-03-06 00:41, Krish Donald wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am new to AWS and would like to setup Hadoop cluster using cloudera
>>>> manager for 6-7 nodes.
>>>>
>>>> t2.micro on AWS; Is it enough for setting up Hadoop cluster ?
>>>> I would like to use free service as of now.
>>>>
>>>> Please advise.
>>>>
>>>> Thanks
>>>> Krish
>
>