hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "叶双明" <yeshuangm...@gmail.com>
Subject Re: How to manage a large cluster?
Date Tue, 16 Sep 2008 00:48:49 GMT
Sorry, but I can't open it:

2008/9/13 Steve Loughran <stevel@apache.org>

> James Moore wrote:
>> On Thu, Sep 11, 2008 at 5:46 AM, Allen Wittenauer <aw@yahoo-inc.com>
>> wrote:
>>> On 9/11/08 2:39 AM, "Alex Loddengaard" <alexlod@google.com> wrote:
>>>> I've never dealt with a large cluster, though I'd imagine it is managed
>>>> the
>>>> same way as small clusters:
>>>   Maybe. :)
> Depends how often you like to be paged, doesn't it :)
>>    Instead, use a real system configuration management package such as
>>> bcfg2, smartfrog, puppet, cfengine, etc.  [Steve, you owe me for the
>>> plug.
>>> :) ]
> Yes Allen, I owe you beer at the next apachecon we are both at.
> Actually, I think Y! were one of the sponsors at the UK event, so we owe
> you for that too.
>  Or on EC2 and its competitors, just build a new image whenever you
>> need to update Hadoop itself.
> 1. It's still good to have as much automation of your image build as you
> can; if you can build new machine images on demand you have have fun/make a
> mess of things. Look at http://instalinux.com to see the web GUI for
> creating linux images on demand that is used inside HP.
> 2. When you try and bring up everything from scratch, you have a
> choreography problem. DNS needs to be up early, and your authentication
> system, the management tools, then the other parts of the system. If you
> have a project where hadoop is integrated with the front end site, for
> example, you're app servers have to stay offline until HDFS is live. So it
> does get complex.
> 3. The Hadoop nodes are good here in that you aren't required to bring up
> the namenode first; the datanodes will wait; same for the task trackers and
> job tracker. But if you, say, need to point everything at a new hostname for
> the namenode, well, that's a config change that needs to be pushed out,
> somehow.
> I'm adding some stuff on different ways to deploy hadoop here:
> http://wiki.smartfrog.org/wiki/display/sf/Patterns+of+Hadoop+Deployment
> -steve

Sorry for my english!! 明
Please help me to correct my english expression and error in syntax
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message