ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Speidel <jspei...@hortonworks.com>
Subject Re: About extensibility of Ambari stacks (not inheritance)
Date Thu, 04 Dec 2014 21:19:50 GMT
Hi Cos,
As mentioned above, it isn't currently possible to augment an existing
stack at runtime with new services/components.  You could possibly change
the stack associated with a running cluster to a new stack which contains
the required services/components, but this new stack definition would need
to be available to ambari so it really isn't very dynamic.  We understand
that going forward that the stack framework will need to be more extensible
and dynamic as additional stack definitions are created.  Some of the
underlying work to make stack processing more flexible has already been
done in anticipation of new requirements in this area.

This is actually the second time that this scenario has came up for me
today so it would be nice to get a Jira filed for an enhancement where the
community can start to discuss the associated use cases.
-John

On Tue, Dec 2, 2014 at 2:00 PM, Konstantin Boudnik <cos@apache.org> wrote:

> Thanks for chiming in, Erin! Do you know if there's any plan to have
> dynamic
> extensibility for services? For instance, Cloudera Manager has this
> concept of
> Custom Services. And while it was done as an added late fix for parcels, it
> provides a way of adding new software components into an existing Hadoop
> cluster. Which seems to be a pretty handy concept, considering the liquid
> state of the whole Hadoop ecosystem.
>
> It seems that this functionality, if desired, is going to be brand new and
> as
> such require not just development but also re-consideration at the design
> level. Am I right? Appreciate the thoughts!
>
> The best part of active-active masters paradigm is that a client can work
> with
> either of them without a concern of which one is ahead or behind of the
> rest
> as they are equal. Essentially this solves some major issues that pester
> active-standby solutions.
>
> And yes, we do install the services to the cluster's nodes as you
> suggested in
> the later part of your email - after all they are just Linux daemons
> (although
> Ambari tries to hide the fact for whatever reason that is). However this
> presents the core of the very issue: Ambari isn't aware about services
> installed outside of its realm, and it doesn't provide any monitoring or
> life-cycle management for those. And because of that I am trying to figure
> out
> what can be done in this regards to have this dynamism.
>
> With regards,
>   Cos
>
> On Tue, Dec 02, 2014 at 08:41AM, Erin Boyd wrote:
> > Hi Cos,
> > Currently, adding services dynamically is not supported in Ambari.  It is
> > generally done, as you mentioned through the creation of a stack with the
> > services definition defined within in.  I also don't believe we currently
> > support multiple masters on the same cluster.  How would one
> deferientiate
> > which master it should be using in such an environment?
> >
> > Of course services can be installed independent of Ambari on the hosts
> > inside the cluster...but I don't think that is what you are getting at.
> Are
> > you expecting the services to have a presence on the UI though installed
> > outside of Ambari?
> >
> > Erin
> >
> > ----- Original Message -----
> > From: "Konstantin Boudnik" <cos@apache.org>
> > To: dev@ambari.apache.org
> > Sent: Monday, December 1, 2014 6:42:30 PM
> > Subject: About extensibility of Ambari stacks (not inheritance)
> >
> > Guys,
> >
> > I am looking into possible stack extensibility properties of Ambari (not
> to be
> > confused with inheritance), but haven't been able to derive any final
> > conclusions just yet. Hence, I'd appreciate the input from the people
> behind
> > the system.
> >
> > I have a few questions about current state of the Ambari (version 1.6.1
> and
> > coming 1.7, and possibly later?) with regards to ability to expand an
> existing
> > stack definitions with a 3rd party services and do it in the runtime,
> rather
> > than only during the installation. We need to be able to run a multiple
> > instances of our master service in the cluster, which isn't typical for
> > "normal" Hadoop concept where only one master can exist for any giving
> > service.
> >
> > Our use case is to be able to amend an exiting cluster setup (HDP,
> Bigtop,
> > etc.) with a new service running on top of HDFS; but not to reinstall the
> > whole stack. The reason we have the use case is that oftentimes our
> software
> > is being added to an existing 3rd party environment, as an added bonus,
> not
> > available during the initial planning and setup of the cluster.
> >
> > The way I understand Ambari's stack inheritance is that a scion stack
> will
> > be a brand new entity, e.g. I won't be able to cherry-pick services from
> it
> > and add them to the already installed parent stack's cluster, right?
> >
> > So far from I see it doesn't seem possible without introducing a
> brand-new
> > version of a stack e.g. 'stack inheritance'. Which, unfortunately, won't
> work
> > as per my explanation of the use above case. I guess another way to look
> at it
> > is this: would it be possible to add a component (or a service) that will
> > override an existing component or a service, but without a need to
> reinstall
> > the rest of the stack?
> >
> > Thanks in advance for any info/ideas.
> >   Cos
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message