brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John McCabe <j...@johnmccabe.net>
Subject Re: YAML blueprint: install package utility
Date Tue, 24 May 2016 08:36:52 GMT
Is there any way we could get rid of the initializers/type parts, rather
than:

- type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
  brooklyn.initializers:
  - type: org.apache.brooklyn.initializer.stock.PackageInstaller
    packages:
      yum: curl
      apt: curl-whatever=2.3.4

Something like:

- type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
  packages:
    yum: curl
    apt: curl-whatever=2.3.4


On Tue, 24 May 2016 at 09:34 Duncan Grant <duncan.grant@cloudsoftcorp.com>
wrote:

> Andrew,
>
> good point.  That's much neater.  Should be pretty straightforward to
> implement as well.
>
> Duncan
>
> On Mon, 23 May 2016 at 21:16 Andrew Kennedy <
> andrew.kennedy@cloudsoftcorp.com> wrote:
>
> > Duncan,
> >
> > I like the idea here, of having some extra object that does the package
> > installation, but what about a customizer or initializer instead? You
> could
> > have something like this:
> >
> > - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
> >   brooklyn.initializers:
> >   - type: org.apache.brooklyn.initializer.stock.PackageInstaller
> >     packages:
> >       yum: curl
> >       apt: curl-whatever=2.3.4
> >
> > I initially thought of a JcloudsLocationCustomizer but that fails when we
> > use BYON locations, so I am thinking of something more like a generic
> > entity customizer that runs after the entity has been created, and the
> > location is available and SSHable. There are lots of other things these
> > initializers could do, like setting up users, configuring networking,
> > mounting volumes and so on. I don't think its a big stretch to add this
> > functionality here.
> >
> > Andrew.
> >
> > On Fri, 6 May 2016 at 14:40 Duncan Grant <duncan.grant@cloudsoftcorp.com
> >
> > wrote:
> >
> > > I agree that this is a problem that needs solved but I have misgivings
> > > about each of the proposed solutions.
> > >
> > > Firstly I think that "discovering" how to write brooklyn yaml can be
> > > difficult.  It can be difficult to find things in the documentation and
> > if
> > > you don't know to look for them in the first place then your only hope
> is
> > > coming across an example.
> > >
> > > So if we pre-install a set of bash scripts before running the install
> > > scripts I don't see any obvious way for someone using brooklyn to find
> > out
> > > about those scripts, and when the scripts change I imagine people won't
> > > notice the new functionality either.
> > > Another downside to using scripts is that it will become more difficult
> > to
> > > debug script failures.  At the moment I can copy the contents of stdin
> > and
> > > see the error associated.  If I had to step into scripts in other files
> > it
> > > could become much harder to find a failure.  In my opinion this ease of
> > > reproducibility and seeing the code I am running is one of the
> advantages
> > > of using bash over just using chef/puppet/salt/etc.
> > > If we could find a widely used, well tested bash library then some of
> > these
> > > issues might be mitigated but I've yet to find one.
> > >
> > > There is a similar issue with discovering the behaviour of the DSL
> > option.
> > > I think that the DSL only works because we keep it to a minimum and
> > really
> > > it only does one task - which is run-time configuring relationships
> > between
> > > entities.  If we extend it then we have replaced java with dsl and I
> > don't
> > > think that benefits anyone.  One advantage of the DSL option is that it
> > > will generate bash so it shouldn't make debugging any harder.
> > >
> > > The advantage of adding config options to the VanillaSoftwareProcess is
> > > that it adds to extra ways of discovering the behaviour - through the
> > > composer's autocomplete and in javadoc.  I assume that this would
> > generate
> > > extra tasks so we could again debug the bash used and it might make it
> > > easier as we might be able to see exactly which package failed to
> > install.
> > > However the big downside to this is that we are starting to make a sort
> > of
> > > god object which contains all brooklyn functionality in one object -
> i.e.
> > > VanilllaSoftwareProcess and I'm not sure that's where we want to go.
> > >
> > > After all that I don't really have a good answer.
> > >
> > > My not too good answer would be to make more use of the brooklyn child
> > > relationship so that a) at least the yaml could be composed from small
> > > pieces of code b) we could write a InstallPackage entity to simplify
> the
> > > yaml.
> > >
> > > This might look something like:
> > >
> > > - type: brooklyn.entity.basic.VanillaSoftwareProcess
> > >   install.command: curl somefile > somfile.txt
> > >   brooklyn.config:
> > >     children.startable.mode: foreground_early
> > >   brooklyn.children:
> > >   - type: brooklyn.entity.basic.VanillaSoftwareProcess
> > >     install.command: |
> > >         which curl || \
> > >             { sudo apt-get update && sudo apt-get install curl ; }
|| \
> > >             { sudo yum update && sudo yum install curl ; } || \
> > >             { echo WARNING: cannot install curl && exit 1 ; }
> > > or
> > >   - type: brooklyn.entity.basic.InstallPackage
> > >     package: curl
> > > or
> > >   - type: brooklyn.entity.basic.InstallPackage
> > >     packages:
> > >       yum: curl
> > >       apt: someOtherCurl
> > >
> > > This way everything is discoverable through documentation, javadoc, and
> > > autocomplete.  Common requirements like curl are re-usable. Generated
> > bash
> > > is explicit and if curl fails then it will go on fire.
> > > On the other hand it requires longer yaml, isn't as readable, and due
> to
> > > the use of child parent relationships has a bit of a learning hurdle
> > >
> > > Regards
> > >
> > > Duncan
> > >
> > >
> > > On Wed, 4 May 2016 at 17:35 Thomas Bouron <
> > thomas.bouron@cloudsoftcorp.com
> > > >
> > > wrote:
> > >
> > > > Hi Guglielmo.
> > > >
> > > > I'm not sure to follow your idea of "stacks", could you elaborate on
> > that
> > > > please?
> > > >
> > > > Best.
> > > >
> > > > On Tue, 3 May 2016, 13:38 Guglielmo Nigri, <
> > > > guglielmo.nigri@cloudsoftcorp.com> wrote:
> > > >
> > > > > I propose a declarative approach. For example we could add a
> > configKey
> > > to
> > > > > VanillaSoftwareProcess called requiredPackages.
> > > > >
> > > > > This way, one could just specify the prerequisite packages, sort
of
> > > > > dependencies for the software process -- this would also open up
> > > > > interesting possibilities such as “stacks” as bundled dependencies.
> > > > >
> > > > > A declarative approach would also work for Windows (or non-bash
> > > > > environments).
> > > > >
> > > > > Cheers,
> > > > > Guglielmo
> > > > >
> > > > >
> > > > > On 3 May 2016 at 14:28, Andrea Turli <
> andrea.turli@cloudsoftcorp.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Great discussion guys!
> > > > > >
> > > > > > It seems that it is a shared need, and I wanted to discuss with
> you
> > > as
> > > > I
> > > > > > was not sure about my approach.
> > > > > >
> > > > > > Andrew,
> > > > > >
> > > > > > I like your proposal, thanks!
> > > > > >
> > > > > > Thanks,
> > > > > > Andrea
> > > > > >
> > > > > >
> > > > > > On 3 May 2016 at 13:05, Aled Sage <aled.sage@gmail.com>
wrote:
> > > > > >
> > > > > > > I lean towards Andrew's approach, rather than a special
> > > > > > > $brooklyn.installPackage. Note that different distros use
> > different
> > > > > > package
> > > > > > > names sometimes, so the parameters to such a function can
get
> > > > annoying.
> > > > > > >
> > > > > > > I've been hesitant about us going down the road of
> > > > > `brooklyn-commands.sh`
> > > > > > > for achieving portable blueprints. It feels like we are
> > increasing
> > > > the
> > > > > > > overlap with things like Chef, Salt, Ansible and Puppet
(which
> we
> > > can
> > > > > > build
> > > > > > > on top of with Brooklyn).
> > > > > > >
> > > > > > > However, if we keep brooklyn-commands.sh small and focused,
> then
> > > > maybe
> > > > > > > it's a good idea.
> > > > > > >
> > > > > > > Aled
> > > > > > >
> > > > > > > p.s. I want to decrease the use of
> > > > `BashCommands.installExecutable()`.
> > > > > > The
> > > > > > > bash command it generates, with all the alternatives and
> > > parentheses,
> > > > > > does
> > > > > > > not look like nice Bash. A brooklyn-commands.sh could do
it
> much
> > > > > cleaner,
> > > > > > > with a properly structured multi-line if...elif... (or
> whatever).
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 03/05/2016 11:43, Andrew Kennedy wrote:
> > > > > > >
> > > > > > >> How about an alternative approach - for all SoftwareProcess
> > > > entities,
> > > > > > >> Brooklyn will copy over a script file called
> > > `brooklyn-commands.sh`
> > > > > and
> > > > > > >> the
> > > > > > >> entity commands will soutce this script before running
the
> rest
> > of
> > > > > their
> > > > > > >> configured commands. The script will contain OS agnostic
> > functions
> > > > > > written
> > > > > > >> in Bash that do things like install packages, download
Curl,
> > get a
> > > > > file
> > > > > > >> from a URL etc. Then, the install config might look
like this:
> > > > > > >>
> > > > > > >> ```
> > > > > > >> install.commands: |
> > > > > > >>    brooklyn-installPackages apt:openjdk-1.8.0
> > > > > > yum:java-1.8.0-openjdk-devel
> > > > > > >> java-1.8.0
> > > > > > >>    brooklyn-installPackages curl
> > > > > > >>    brooklyn-runAsRoot cp /tmp/whatever /etc/hosts
> > > > > > >> ```
> > > > > > >>
> > > > > > >> Andrew.
> > > > > > >>
> > > > > > >> On Tue, 3 May 2016 at 11:24 Andrea Turli <
> > > > > > andrea.turli@cloudsoftcorp.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> hi,
> > > > > > >>>
> > > > > > >>> I’ve been thinking about an utility to simplify
the YAML
> > > blueprint
> > > > > > >>> creation: from my experience when using
> VanillaSoftwareProcess
> > is
> > > > > > >>> annoying
> > > > > > >>> to write a portable script just to install a package
(say,
> > java)
> > > > > valid
> > > > > > >>> for
> > > > > > >>> apt, yum, etc so I usually write it (multiple times)
just for
> > an
> > > > OS.
> > > > > > >>>
> > > > > > >>> To increase the portability of the YAML blueprint
I’d like to
> > > > suggest
> > > > > > we
> > > > > > >>> extend the brooklyn DSL with something like:
> > > > > > >>> ```
> > > > > > >>> $brooklyn.installPackage(“curl”)
> > > > > > >>> $brooklyn:installPackage({"apt", "openjdk-1.8.0",
"yum",
> > > > > > >>> "java-1.8.0-openjdk-devel"}, "java-1.8.0")
> > > > > > >>> ```
> > > > > > >>> instead of things like
> > > > > > >>> ```
> > > > > > >>> which curl || \
> > > > > > >>>          { sudo apt-get update && sudo
apt-get install curl
> ; }
> > > ||
> > > > \
> > > > > > >>>          { sudo yum update && sudo yum
install curl ; } || \
> > > > > > >>>          { echo WARNING: cannot install curl &&
exit 1 ; }
> > > > > > >>> ```
> > > > > > >>> I’m not entirely sure this feature fits well
on the DSL.
> > > > > > >>>
> > > > > > >>> Alternatively, we could add a configKey to
> > VanillaSoftwareProcess
> > > > > > called
> > > > > > >>> requiredPackages for a more declarative approach
> (@googlielmo's
> > > > idea)
> > > > > > >>>
> > > > > > >>> Wdyt?
> > > > > > >>>
> > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > > Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
> > > > http://www.cloudsoftcorp.com/
> > > > Github: https://github.com/tbouron
> > > > Twitter: https://twitter.com/eltibouron
> > > >
> > >
> > --
> >
> > Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message