hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Denissov <adenis...@pivotal.io>
Subject Re: hawq Ambari integration
Date Fri, 03 Mar 2017 01:17:38 GMT
Ambari is designed as a tool to sit on top of individual service CLI tools
and provide consistent user experience. Ambari can also have wizards and
custom actions that go beyond service lifecycle management. It is assumed
by Ambari design that once Ambari is used to manage configurations, CLI
tools should not be used, otherwise changes will be out-of-sync or will be
overwritten by Ambari upon service restart.

Having HAWQ CLI tools communicate back to Ambari is not desirable because:
- knowledge of Ambari REST APIs needs to be built into the tools
- Ambari APIs change outside of service release lifecycle
- complexity in error handling (what do you do if Ambari call failed ?)
- introduction of feedback loop and potential infinite cycle (Ambari calls
the tool, the tool calls Ambari back, etc)

If customers want to do some extra automation they can decide how to tie
tools and Ambari, if they like, but I will say we should not have HAWQ CLI
tools call Ambari.

--
Thanks,
Alex Denissov

On Thu, Mar 2, 2017 at 4:44 PM, Alex (Oleksandr) Diachenko <
odiachenko@pivotal.io> wrote:

> Sure,
>
> Users can automate configuration changes via Ambari API, which provides
> unified access
> to all services in a cluster. It's very likely they need to configure HAWQ,
> HDFS and YARN
> when working with configuration changes, so with Ambari API it would be
> easier to call one API
> rather than using all different CLI's.
>
> Regards, Alex.
>
> On Thu, Mar 2, 2017 at 4:39 PM, Lei Chang <lei_chang@apache.org> wrote:
>
> > I think there are some use cases we need What Jon proposed.
> >
> > For example, users installed hawq via Ambari, but they want to automate
> the
> > configuration changes, it would be convenient to have hawq CLI change the
> > configurations.
> >
> > Cheers
> > Lei
> >
> >
> >
> >
> >
> >
> >
> > On Fri, Mar 3, 2017 at 8:36 AM, Alex (Oleksandr) Diachenko <
> > odiachenko@pivotal.io> wrote:
> >
> > > I see, that makes sense.
> > > But is there any action users cannot do via Ambari?
> > >
> > > Ranger is also a good example, there we are making assumption,
> > > users either use Ranger or HAWQ's authorization engine.
> > >
> > > The same logic might be extrapolated to HAWQ/Ambari - users might use
> > > either Ambari or HAWQ CLI, but not both at the same time.
> > > In that way, we can keep things simple.
> > >
> > > Regards, Alex.
> > >
> > >
> > >
> > > On Thu, Mar 2, 2017 at 4:28 PM, Jon Roberts <jroberts@pivotal.io>
> wrote:
> > >
> > > > Right.  Just like HAWQ will be operational without Ranger.
> > > >
> > > > We have the hawq CLI and will obviously continue to have it.  Some
> > people
> > > > use Ambari while others don't.  So just like with Ranger support,
> > > integrate
> > > > when possible but don't require it.
> > > >
> > > >
> > > > Jon
> > > >
> > > > On Thu, Mar 2, 2017 at 6:26 PM, Alex (Oleksandr) Diachenko <
> > > > odiachenko@pivotal.io> wrote:
> > > >
> > > > > Not really, because HAWQ should be operational even without
> Ambari(if
> > > > > that's the case).
> > > > >
> > > > > On Thu, Mar 2, 2017 at 4:21 PM, Jon Roberts <jroberts@pivotal.io>
> > > wrote:
> > > > >
> > > > > > If that is the case, should we remove the "hawq" CLI?
> > > > > >
> > > > > > Jon
> > > > > >
> > > > > > On Thu, Mar 2, 2017 at 6:12 PM, Alex (Oleksandr) Diachenko <
> > > > > > odiachenko@pivotal.io> wrote:
> > > > > >
> > > > > > > Hi Jon,
> > > > > > >
> > > > > > > I think it was designed that Ambari is supposed to be only
one
> > > source
> > > > > of
> > > > > > > true.
> > > > > > > The whole purpose of integration id to provide a user-friendly
> > > > > interface
> > > > > > > and avoid manually editing/distributing config files
> > > > > > > or running CLI commands.
> > > > > > > The idea of coupling HAWQ master with Ambari doesn't seem
to be
> > > > clean.
> > > > > > >
> > > > > > > Regards, Alex.
> > > > > > >
> > > > > > > On Thu, Mar 2, 2017 at 4:05 PM, Jon Roberts <
> jroberts@pivotal.io
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > It would be handy if the "hawq config" also updated
Ambari's
> > > > database
> > > > > > so
> > > > > > > > that changes could be made in either place are retained
when
> > > > changes
> > > > > > are
> > > > > > > > made in either place.
> > > > > > > >
> > > > > > > > Register Ambari:
> > > > > > > > hawq ambari -u admin -w admin -h myhost -p 8080
> > > > > > > >
> > > > > > > > "hawq config" could then raise INFO/WARN messages
about
> > updating
> > > > > > Ambari.
> > > > > > > >
> > > > > > > > Example:
> > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb
> > > > > > > > INFO: Updated Ambari with hawq_rm_stmt_vseg_memory=16gb
> > > > > > > > or
> > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb
> > > > > > > > WARN: Failed to update Ambari with
> > hawq_rm_stmt_vseg_memory=16gb.
> > > > > > Please
> > > > > > > > update Ambari credentials manually to retain this
> configuration
> > > > > change
> > > > > > > > after a restart.
> > > > > > > >
> > > > > > > > The implementation would require interacting with
the Ambari
> > APIs
> > > > and
> > > > > > > also
> > > > > > > > storing the credentials in an encrypted file on the
HAWQ
> > Master.
> > > > > > > >
> > > > > > > > Thoughts?
> > > > > > > >
> > > > > > > >
> > > > > > > > Jon Roberts
> > > > > > > > Principal Engineer | jroberts@pivotal.io | 615-426-8661
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message