brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Bouron <thomas.bou...@cloudsoftcorp.com>
Subject Re: YAML blueprint: install package utility
Date Tue, 24 May 2016 08:57:01 GMT
I quite like Andrew's approach using an initializer, feels elegant and
relatively easy to implement.

Regarding the syntax, getting a rid of the `brooklyn.initializer` and
`type` is good for a discoverability point of view but that means going
through a config key which feels wrong in that case. I don't have an
argument for it, it's more a gut feeling.

On Tue, 24 May 2016 at 09:37 John McCabe <john@johnmccabe.net> wrote:

> Is there any way we could get rid of the initializers/type parts, rather
> than:
>
> - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
>   brooklyn.initializers:
>   - type: org.apache.brooklyn.initializer.stock.PackageInstaller
>     packages:
>       yum: curl
>       apt: curl-whatever=2.3.4
>
> Something like:
>
> - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
>   packages:
>     yum: curl
>     apt: curl-whatever=2.3.4
>
>
> On Tue, 24 May 2016 at 09:34 Duncan Grant <duncan.grant@cloudsoftcorp.com>
> wrote:
>
> > Andrew,
> >
> > good point.  That's much neater.  Should be pretty straightforward to
> > implement as well.
> >
> > Duncan
> >
> > On Mon, 23 May 2016 at 21:16 Andrew Kennedy <
> > andrew.kennedy@cloudsoftcorp.com> wrote:
> >
> > > Duncan,
> > >
> > > I like the idea here, of having some extra object that does the package
> > > installation, but what about a customizer or initializer instead? You
> > could
> > > have something like this:
> > >
> > > - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
> > >   brooklyn.initializers:
> > >   - type: org.apache.brooklyn.initializer.stock.PackageInstaller
> > >     packages:
> > >       yum: curl
> > >       apt: curl-whatever=2.3.4
> > >
> > > I initially thought of a JcloudsLocationCustomizer but that fails when
> we
> > > use BYON locations, so I am thinking of something more like a generic
> > > entity customizer that runs after the entity has been created, and the
> > > location is available and SSHable. There are lots of other things these
> > > initializers could do, like setting up users, configuring networking,
> > > mounting volumes and so on. I don't think its a big stretch to add this
> > > functionality here.
> > >
> > > Andrew.
> > >
> > > On Fri, 6 May 2016 at 14:40 Duncan Grant <
> duncan.grant@cloudsoftcorp.com
> > >
> > > wrote:
> > >
> > > > I agree that this is a problem that needs solved but I have
> misgivings
> > > > about each of the proposed solutions.
> > > >
> > > > Firstly I think that "discovering" how to write brooklyn yaml can be
> > > > difficult.  It can be difficult to find things in the documentation
> and
> > > if
> > > > you don't know to look for them in the first place then your only
> hope
> > is
> > > > coming across an example.
> > > >
> > > > So if we pre-install a set of bash scripts before running the install
> > > > scripts I don't see any obvious way for someone using brooklyn to
> find
> > > out
> > > > about those scripts, and when the scripts change I imagine people
> won't
> > > > notice the new functionality either.
> > > > Another downside to using scripts is that it will become more
> difficult
> > > to
> > > > debug script failures.  At the moment I can copy the contents of
> stdin
> > > and
> > > > see the error associated.  If I had to step into scripts in other
> files
> > > it
> > > > could become much harder to find a failure.  In my opinion this ease
> of
> > > > reproducibility and seeing the code I am running is one of the
> > advantages
> > > > of using bash over just using chef/puppet/salt/etc.
> > > > If we could find a widely used, well tested bash library then some of
> > > these
> > > > issues might be mitigated but I've yet to find one.
> > > >
> > > > There is a similar issue with discovering the behaviour of the DSL
> > > option.
> > > > I think that the DSL only works because we keep it to a minimum and
> > > really
> > > > it only does one task - which is run-time configuring relationships
> > > between
> > > > entities.  If we extend it then we have replaced java with dsl and I
> > > don't
> > > > think that benefits anyone.  One advantage of the DSL option is that
> it
> > > > will generate bash so it shouldn't make debugging any harder.
> > > >
> > > > The advantage of adding config options to the VanillaSoftwareProcess
> is
> > > > that it adds to extra ways of discovering the behaviour - through the
> > > > composer's autocomplete and in javadoc.  I assume that this would
> > > generate
> > > > extra tasks so we could again debug the bash used and it might make
> it
> > > > easier as we might be able to see exactly which package failed to
> > > install.
> > > > However the big downside to this is that we are starting to make a
> sort
> > > of
> > > > god object which contains all brooklyn functionality in one object -
> > i.e.
> > > > VanilllaSoftwareProcess and I'm not sure that's where we want to go.
> > > >
> > > > After all that I don't really have a good answer.
> > > >
> > > > My not too good answer would be to make more use of the brooklyn
> child
> > > > relationship so that a) at least the yaml could be composed from
> small
> > > > pieces of code b) we could write a InstallPackage entity to simplify
> > the
> > > > yaml.
> > > >
> > > > This might look something like:
> > > >
> > > > - type: brooklyn.entity.basic.VanillaSoftwareProcess
> > > >   install.command: curl somefile > somfile.txt
> > > >   brooklyn.config:
> > > >     children.startable.mode: foreground_early
> > > >   brooklyn.children:
> > > >   - type: brooklyn.entity.basic.VanillaSoftwareProcess
> > > >     install.command: |
> > > >         which curl || \
> > > >             { sudo apt-get update && sudo apt-get install curl
; }
> || \
> > > >             { sudo yum update && sudo yum install curl ; } ||
\
> > > >             { echo WARNING: cannot install curl && exit 1 ; }
> > > > or
> > > >   - type: brooklyn.entity.basic.InstallPackage
> > > >     package: curl
> > > > or
> > > >   - type: brooklyn.entity.basic.InstallPackage
> > > >     packages:
> > > >       yum: curl
> > > >       apt: someOtherCurl
> > > >
> > > > This way everything is discoverable through documentation, javadoc,
> and
> > > > autocomplete.  Common requirements like curl are re-usable. Generated
> > > bash
> > > > is explicit and if curl fails then it will go on fire.
> > > > On the other hand it requires longer yaml, isn't as readable, and due
> > to
> > > > the use of child parent relationships has a bit of a learning hurdle
> > > >
> > > > Regards
> > > >
> > > > Duncan
> > > >
> > > >
> > > > On Wed, 4 May 2016 at 17:35 Thomas Bouron <
> > > thomas.bouron@cloudsoftcorp.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Guglielmo.
> > > > >
> > > > > I'm not sure to follow your idea of "stacks", could you elaborate
> on
> > > that
> > > > > please?
> > > > >
> > > > > Best.
> > > > >
> > > > > On Tue, 3 May 2016, 13:38 Guglielmo Nigri, <
> > > > > guglielmo.nigri@cloudsoftcorp.com> wrote:
> > > > >
> > > > > > I propose a declarative approach. For example we could add a
> > > configKey
> > > > to
> > > > > > VanillaSoftwareProcess called requiredPackages.
> > > > > >
> > > > > > This way, one could just specify the prerequisite packages,
sort
> of
> > > > > > dependencies for the software process -- this would also open
up
> > > > > > interesting possibilities such as “stacks” as bundled
> dependencies.
> > > > > >
> > > > > > A declarative approach would also work for Windows (or non-bash
> > > > > > environments).
> > > > > >
> > > > > > Cheers,
> > > > > > Guglielmo
> > > > > >
> > > > > >
> > > > > > On 3 May 2016 at 14:28, Andrea Turli <
> > andrea.turli@cloudsoftcorp.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Great discussion guys!
> > > > > > >
> > > > > > > It seems that it is a shared need, and I wanted to discuss
with
> > you
> > > > as
> > > > > I
> > > > > > > was not sure about my approach.
> > > > > > >
> > > > > > > Andrew,
> > > > > > >
> > > > > > > I like your proposal, thanks!
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Andrea
> > > > > > >
> > > > > > >
> > > > > > > On 3 May 2016 at 13:05, Aled Sage <aled.sage@gmail.com>
wrote:
> > > > > > >
> > > > > > > > I lean towards Andrew's approach, rather than a special
> > > > > > > > $brooklyn.installPackage. Note that different distros
use
> > > different
> > > > > > > package
> > > > > > > > names sometimes, so the parameters to such a function
can get
> > > > > annoying.
> > > > > > > >
> > > > > > > > I've been hesitant about us going down the road of
> > > > > > `brooklyn-commands.sh`
> > > > > > > > for achieving portable blueprints. It feels like we
are
> > > increasing
> > > > > the
> > > > > > > > overlap with things like Chef, Salt, Ansible and Puppet
> (which
> > we
> > > > can
> > > > > > > build
> > > > > > > > on top of with Brooklyn).
> > > > > > > >
> > > > > > > > However, if we keep brooklyn-commands.sh small and
focused,
> > then
> > > > > maybe
> > > > > > > > it's a good idea.
> > > > > > > >
> > > > > > > > Aled
> > > > > > > >
> > > > > > > > p.s. I want to decrease the use of
> > > > > `BashCommands.installExecutable()`.
> > > > > > > The
> > > > > > > > bash command it generates, with all the alternatives
and
> > > > parentheses,
> > > > > > > does
> > > > > > > > not look like nice Bash. A brooklyn-commands.sh could
do it
> > much
> > > > > > cleaner,
> > > > > > > > with a properly structured multi-line if...elif...
(or
> > whatever).
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 03/05/2016 11:43, Andrew Kennedy wrote:
> > > > > > > >
> > > > > > > >> How about an alternative approach - for all SoftwareProcess
> > > > > entities,
> > > > > > > >> Brooklyn will copy over a script file called
> > > > `brooklyn-commands.sh`
> > > > > > and
> > > > > > > >> the
> > > > > > > >> entity commands will soutce this script before
running the
> > rest
> > > of
> > > > > > their
> > > > > > > >> configured commands. The script will contain OS
agnostic
> > > functions
> > > > > > > written
> > > > > > > >> in Bash that do things like install packages,
download Curl,
> > > get a
> > > > > > file
> > > > > > > >> from a URL etc. Then, the install config might
look like
> this:
> > > > > > > >>
> > > > > > > >> ```
> > > > > > > >> install.commands: |
> > > > > > > >>    brooklyn-installPackages apt:openjdk-1.8.0
> > > > > > > yum:java-1.8.0-openjdk-devel
> > > > > > > >> java-1.8.0
> > > > > > > >>    brooklyn-installPackages curl
> > > > > > > >>    brooklyn-runAsRoot cp /tmp/whatever /etc/hosts
> > > > > > > >> ```
> > > > > > > >>
> > > > > > > >> Andrew.
> > > > > > > >>
> > > > > > > >> On Tue, 3 May 2016 at 11:24 Andrea Turli <
> > > > > > > andrea.turli@cloudsoftcorp.com>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> hi,
> > > > > > > >>>
> > > > > > > >>> I’ve been thinking about an utility to simplify
the YAML
> > > > blueprint
> > > > > > > >>> creation: from my experience when using
> > VanillaSoftwareProcess
> > > is
> > > > > > > >>> annoying
> > > > > > > >>> to write a portable script just to install
a package (say,
> > > java)
> > > > > > valid
> > > > > > > >>> for
> > > > > > > >>> apt, yum, etc so I usually write it (multiple
times) just
> for
> > > an
> > > > > OS.
> > > > > > > >>>
> > > > > > > >>> To increase the portability of the YAML blueprint
I’d like
> to
> > > > > suggest
> > > > > > > we
> > > > > > > >>> extend the brooklyn DSL with something like:
> > > > > > > >>> ```
> > > > > > > >>> $brooklyn.installPackage(“curl”)
> > > > > > > >>> $brooklyn:installPackage({"apt", "openjdk-1.8.0",
"yum",
> > > > > > > >>> "java-1.8.0-openjdk-devel"}, "java-1.8.0")
> > > > > > > >>> ```
> > > > > > > >>> instead of things like
> > > > > > > >>> ```
> > > > > > > >>> which curl || \
> > > > > > > >>>          { sudo apt-get update &&
sudo apt-get install curl
> > ; }
> > > > ||
> > > > > \
> > > > > > > >>>          { sudo yum update && sudo
yum install curl ; } ||
> \
> > > > > > > >>>          { echo WARNING: cannot install curl
&& exit 1 ; }
> > > > > > > >>> ```
> > > > > > > >>> I’m not entirely sure this feature fits
well on the DSL.
> > > > > > > >>>
> > > > > > > >>> Alternatively, we could add a configKey to
> > > VanillaSoftwareProcess
> > > > > > > called
> > > > > > > >>> requiredPackages for a more declarative approach
> > (@googlielmo's
> > > > > idea)
> > > > > > > >>>
> > > > > > > >>> Wdyt?
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > --
> > > > > Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
> > > > > http://www.cloudsoftcorp.com/
> > > > > Github: https://github.com/tbouron
> > > > > Twitter: https://twitter.com/eltibouron
> > > > >
> > > >
> > > --
> > >
> > > Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft
> > >
> >
>
-- 

Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
http://www.cloudsoftcorp.com/
Github: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message