hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@veoh.com>
Subject Re: multiple datanodes in the same machine
Date Tue, 15 Apr 2008 20:37:40 GMT

I have had no issues in scaling the number of datanodes.  The location of
the data is almost invisible to MR programs.

I have had issues in going from local to distributed mode, but that has
entirely been due to class path like issues.  Since MR naturally restricts
your focus, it is pretty much the rule that programs scale without much
thought.

If you test with two tasktrackers and one data node, you should have a
pretty solid test environment.


On 4/15/08 1:12 PM, "cagdas.gerede@gmail.com" <cagdas.gerede@gmail.com>
wrote:

> Testing when I do not have 10 machines.
> 
> 
> On 4/15/08, Ted Dunning <tdunning@veoh.com> wrote:
>> 
>> Why do you want to do this perverse thing?
>> 
>> How does it help to have more than one datanode per machine?  And what in
>> the world is better when you have 10?
>> 
>> 
>> On 4/15/08 12:53 PM, "Cagdas Gerede" <cagdas.gerede@gmail.com> wrote:
>> 
>>> I have a follow-up question,
>>> Is there a way to programatically configure datanode parameters and start
>>> the datanode process?
>>> If I want to create 10 datanodes on the same host, do I have to create 10
>>> config files?
>>> 
>>> 
>>> On Tue, Apr 15, 2008 at 12:29 PM, dhruba Borthakur <dhruba@yahoo-inc.com>
>>> wrote:
>>> 
>>>> Yes, just point the Datanodes to different config files, different sets
>>>> of ports, different data directories. Etc.etc.
>>>> 
>>>> Thanks,
>>>> dhruba
>>>> 
>>>> -----Original Message-----
>>>> From: Cagdas Gerede [mailto:cagdas.gerede@gmail.com]
>>>> Sent: Tuesday, April 15, 2008 11:21 AM
>>>> To: core-user@hadoop.apache.org
>>>> Subject: multiple datanodes in the same machine
>>>> 
>>>> Is there a way to run multiple datanodes in the same machine?
>>>> 
>>>> 
>>>> --
>>>> ------------
>>>> Best Regards, Cagdas Evren Gerede
>>>> Home Page: http://cagdasgerede.info
>>>> 
>>> 
>>> 
>> 
>> 
> 


Mime
View raw message