bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Núñez <>
Subject Re: Cluster Management: OpenSource & Vendor Options
Date Wed, 01 Jan 2014 14:50:58 GMT
I can¹t comment on the direction of Ambari, nor CM¹s internals, but a good
set of Hadoop command-line management tools, based on puppet, chef or
salt, would certainly be a good addition to the community, regardless of
BigTop supporting Ambari. Especially since it allows easy integration into
existing data centre administration & operations processes.

If sufficiently mature, a thin GUI shouldn¹t be that hard to add for
common operations. The question is: where does this fit? In BigTop as a
sub-project? A project in and of itself? It¹s not a trivial amount of
work, and Ambari might just be the Œgood enough¹ path of least resistance;
at least being open source some hooks could be added to support
orchestration tools.

	- SteveN

On 2013-12-31 14:21 , "Konstantin Boudnik" <> wrote:

>The reason for not-much activity on that JIRA is because Ambari seems to
>drifting away from real-life orchestration systems like Puppet toward
>something else. And that's exactly why I prefer to use Puppet or Chef
>orchestration - you have a state machine that works in the same way on
>supported platform.
>In 20 years of doing system and network administration as well as software
>development I've seen times and again how the fancy UI applications fail
>deliver on their promise. Main reason is in the shifting of the focus on
>bling instead of the core functionality.
>Cloudera's CM is a perfect example of my point, because it is doing
>heinous things with standard Linux services, their life-cycle and
>configurations up to the point where any sane Hadoop devops would be
>to do anything without CM. Ambrai seems to be a bit better in this
>However, with the development above and replacement of the Puppet I am not
>sure how much longer it will be the case.
>Happy New Year everyone!
>  Cos
>On Tue, Dec 31, 2013 at 03:10PM, Steven Núñez wrote:
>> Thanks. That issue answers pretty much all the questions. I¹d certainly
>> it a +1 if I had a login. That definitely seems like the right
>>direction to
>> move in. I don¹t know the internals, but if everyone is using BigTop for
>> packaging, perhaps there¹s some way to read the manifest files (if
>> what they¹re called) to produce what Ambari needs for management.
>> From: Chris Mildebrandt
>> Reply-To: "<>"
>> Date: Tuesday, 31 December 2013 7:57
>> To: "<>"
>> Cc: "<>"
>> Subject: Re: Cluster Management: OpenSource & Vendor Options
>> You may want to watch this:
>> and include it in your write-up for future considerations. Though there
>>hasn't been much activity on it.
>> On Mon, Dec 30, 2013 at 8:32 AM, Steven Núñez
>><<>> wrote:
>> Seasons Greetings All,
>> I¹m doing a bit of a write-up on the various Hadoop distributions and
>>would like to understand exactly what packages are installed by the
>>Apache version of Ambari. It¹s an exciting place to be working (big data
>>& Hadoop) but the lines are blurred in many ways. The way I see the open
>>source landscape now is something like this (from a
>>management/installation/configuration perspective):
>> BigTop -> RPM like packaging for Hadoop
>> Ambari -> GUI management/monitoring/provisioning
>> Looking at it from a vendor perspective, we¹ve got (I know there are
>>others, this is just for discussion):
>> BigTop (packaging)
>> CDH
>> HDP
>> Apache Bigtop
>> Cloudera
>> Cloudera Manager (closed source, commercial)
>> Hortonworks / Apache
>> Ambari(open source)
>> The CDH, BigTop and HDP (I assume) base distributions require a lot of
>>manual configuration, so the best way to spin up a cluster with a
>>reasonable set of applications (say HDFS, YARN, Hive, HCatalog, HBase,
>>ZooKeeper, Oozie, Pig, Sqoop) is to use CDH + CM or Ambari + HDP.
>> Is there an equivalent for Apache? If I use the kit found at
>><> to spin up a cluster, do I
>>get Apache components, or the HDP distribution? I¹m trying to define the
>>ŒApache distribution¹ in my mind, if there is one, and understand
>>exactly what its capabilities are, and cluster management is rather
>>fundamental, since not many folks have the luxury of spending time
>>climbing the long, steep learning curve of Hadoop ecosystem
>> Cheers,
>> - SteveN

View raw message