brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Kennedy <andrew.kenn...@cloudsoftcorp.com>
Subject Re: YAML blueprint: install package utility
Date Mon, 23 May 2016 20:15:54 GMT
Duncan,

I like the idea here, of having some extra object that does the package
installation, but what about a customizer or initializer instead? You could
have something like this:

- type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
  brooklyn.initializers:
  - type: org.apache.brooklyn.initializer.stock.PackageInstaller
    packages:
      yum: curl
      apt: curl-whatever=2.3.4

I initially thought of a JcloudsLocationCustomizer but that fails when we
use BYON locations, so I am thinking of something more like a generic
entity customizer that runs after the entity has been created, and the
location is available and SSHable. There are lots of other things these
initializers could do, like setting up users, configuring networking,
mounting volumes and so on. I don't think its a big stretch to add this
functionality here.

Andrew.

On Fri, 6 May 2016 at 14:40 Duncan Grant <duncan.grant@cloudsoftcorp.com>
wrote:

> I agree that this is a problem that needs solved but I have misgivings
> about each of the proposed solutions.
>
> Firstly I think that "discovering" how to write brooklyn yaml can be
> difficult.  It can be difficult to find things in the documentation and if
> you don't know to look for them in the first place then your only hope is
> coming across an example.
>
> So if we pre-install a set of bash scripts before running the install
> scripts I don't see any obvious way for someone using brooklyn to find out
> about those scripts, and when the scripts change I imagine people won't
> notice the new functionality either.
> Another downside to using scripts is that it will become more difficult to
> debug script failures.  At the moment I can copy the contents of stdin and
> see the error associated.  If I had to step into scripts in other files it
> could become much harder to find a failure.  In my opinion this ease of
> reproducibility and seeing the code I am running is one of the advantages
> of using bash over just using chef/puppet/salt/etc.
> If we could find a widely used, well tested bash library then some of these
> issues might be mitigated but I've yet to find one.
>
> There is a similar issue with discovering the behaviour of the DSL option.
> I think that the DSL only works because we keep it to a minimum and really
> it only does one task - which is run-time configuring relationships between
> entities.  If we extend it then we have replaced java with dsl and I don't
> think that benefits anyone.  One advantage of the DSL option is that it
> will generate bash so it shouldn't make debugging any harder.
>
> The advantage of adding config options to the VanillaSoftwareProcess is
> that it adds to extra ways of discovering the behaviour - through the
> composer's autocomplete and in javadoc.  I assume that this would generate
> extra tasks so we could again debug the bash used and it might make it
> easier as we might be able to see exactly which package failed to install.
> However the big downside to this is that we are starting to make a sort of
> god object which contains all brooklyn functionality in one object - i.e.
> VanilllaSoftwareProcess and I'm not sure that's where we want to go.
>
> After all that I don't really have a good answer.
>
> My not too good answer would be to make more use of the brooklyn child
> relationship so that a) at least the yaml could be composed from small
> pieces of code b) we could write a InstallPackage entity to simplify the
> yaml.
>
> This might look something like:
>
> - type: brooklyn.entity.basic.VanillaSoftwareProcess
>   install.command: curl somefile > somfile.txt
>   brooklyn.config:
>     children.startable.mode: foreground_early
>   brooklyn.children:
>   - type: brooklyn.entity.basic.VanillaSoftwareProcess
>     install.command: |
>         which curl || \
>             { sudo apt-get update && sudo apt-get install curl ; } || \
>             { sudo yum update && sudo yum install curl ; } || \
>             { echo WARNING: cannot install curl && exit 1 ; }
> or
>   - type: brooklyn.entity.basic.InstallPackage
>     package: curl
> or
>   - type: brooklyn.entity.basic.InstallPackage
>     packages:
>       yum: curl
>       apt: someOtherCurl
>
> This way everything is discoverable through documentation, javadoc, and
> autocomplete.  Common requirements like curl are re-usable. Generated bash
> is explicit and if curl fails then it will go on fire.
> On the other hand it requires longer yaml, isn't as readable, and due to
> the use of child parent relationships has a bit of a learning hurdle
>
> Regards
>
> Duncan
>
>
> On Wed, 4 May 2016 at 17:35 Thomas Bouron <thomas.bouron@cloudsoftcorp.com
> >
> wrote:
>
> > Hi Guglielmo.
> >
> > I'm not sure to follow your idea of "stacks", could you elaborate on that
> > please?
> >
> > Best.
> >
> > On Tue, 3 May 2016, 13:38 Guglielmo Nigri, <
> > guglielmo.nigri@cloudsoftcorp.com> wrote:
> >
> > > I propose a declarative approach. For example we could add a configKey
> to
> > > VanillaSoftwareProcess called requiredPackages.
> > >
> > > This way, one could just specify the prerequisite packages, sort of
> > > dependencies for the software process -- this would also open up
> > > interesting possibilities such as “stacks” as bundled dependencies.
> > >
> > > A declarative approach would also work for Windows (or non-bash
> > > environments).
> > >
> > > Cheers,
> > > Guglielmo
> > >
> > >
> > > On 3 May 2016 at 14:28, Andrea Turli <andrea.turli@cloudsoftcorp.com>
> > > wrote:
> > >
> > > > Great discussion guys!
> > > >
> > > > It seems that it is a shared need, and I wanted to discuss with you
> as
> > I
> > > > was not sure about my approach.
> > > >
> > > > Andrew,
> > > >
> > > > I like your proposal, thanks!
> > > >
> > > > Thanks,
> > > > Andrea
> > > >
> > > >
> > > > On 3 May 2016 at 13:05, Aled Sage <aled.sage@gmail.com> wrote:
> > > >
> > > > > I lean towards Andrew's approach, rather than a special
> > > > > $brooklyn.installPackage. Note that different distros use different
> > > > package
> > > > > names sometimes, so the parameters to such a function can get
> > annoying.
> > > > >
> > > > > I've been hesitant about us going down the road of
> > > `brooklyn-commands.sh`
> > > > > for achieving portable blueprints. It feels like we are increasing
> > the
> > > > > overlap with things like Chef, Salt, Ansible and Puppet (which we
> can
> > > > build
> > > > > on top of with Brooklyn).
> > > > >
> > > > > However, if we keep brooklyn-commands.sh small and focused, then
> > maybe
> > > > > it's a good idea.
> > > > >
> > > > > Aled
> > > > >
> > > > > p.s. I want to decrease the use of
> > `BashCommands.installExecutable()`.
> > > > The
> > > > > bash command it generates, with all the alternatives and
> parentheses,
> > > > does
> > > > > not look like nice Bash. A brooklyn-commands.sh could do it much
> > > cleaner,
> > > > > with a properly structured multi-line if...elif... (or whatever).
> > > > >
> > > > >
> > > > >
> > > > > On 03/05/2016 11:43, Andrew Kennedy wrote:
> > > > >
> > > > >> How about an alternative approach - for all SoftwareProcess
> > entities,
> > > > >> Brooklyn will copy over a script file called
> `brooklyn-commands.sh`
> > > and
> > > > >> the
> > > > >> entity commands will soutce this script before running the rest
of
> > > their
> > > > >> configured commands. The script will contain OS agnostic functions
> > > > written
> > > > >> in Bash that do things like install packages, download Curl,
get a
> > > file
> > > > >> from a URL etc. Then, the install config might look like this:
> > > > >>
> > > > >> ```
> > > > >> install.commands: |
> > > > >>    brooklyn-installPackages apt:openjdk-1.8.0
> > > > yum:java-1.8.0-openjdk-devel
> > > > >> java-1.8.0
> > > > >>    brooklyn-installPackages curl
> > > > >>    brooklyn-runAsRoot cp /tmp/whatever /etc/hosts
> > > > >> ```
> > > > >>
> > > > >> Andrew.
> > > > >>
> > > > >> On Tue, 3 May 2016 at 11:24 Andrea Turli <
> > > > andrea.turli@cloudsoftcorp.com>
> > > > >> wrote:
> > > > >>
> > > > >> hi,
> > > > >>>
> > > > >>> I’ve been thinking about an utility to simplify the YAML
> blueprint
> > > > >>> creation: from my experience when using VanillaSoftwareProcess
is
> > > > >>> annoying
> > > > >>> to write a portable script just to install a package (say,
java)
> > > valid
> > > > >>> for
> > > > >>> apt, yum, etc so I usually write it (multiple times) just
for an
> > OS.
> > > > >>>
> > > > >>> To increase the portability of the YAML blueprint I’d like
to
> > suggest
> > > > we
> > > > >>> extend the brooklyn DSL with something like:
> > > > >>> ```
> > > > >>> $brooklyn.installPackage(“curl”)
> > > > >>> $brooklyn:installPackage({"apt", "openjdk-1.8.0", "yum",
> > > > >>> "java-1.8.0-openjdk-devel"}, "java-1.8.0")
> > > > >>> ```
> > > > >>> instead of things like
> > > > >>> ```
> > > > >>> which curl || \
> > > > >>>          { sudo apt-get update && sudo apt-get install
curl ; }
> ||
> > \
> > > > >>>          { sudo yum update && sudo yum install curl
; } || \
> > > > >>>          { echo WARNING: cannot install curl && exit
1 ; }
> > > > >>> ```
> > > > >>> I’m not entirely sure this feature fits well on the DSL.
> > > > >>>
> > > > >>> Alternatively, we could add a configKey to VanillaSoftwareProcess
> > > > called
> > > > >>> requiredPackages for a more declarative approach (@googlielmo's
> > idea)
> > > > >>>
> > > > >>> Wdyt?
> > > > >>>
> > > > >>>
> > > > >
> > > >
> > >
> > --
> > Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
> > http://www.cloudsoftcorp.com/
> > Github: https://github.com/tbouron
> > Twitter: https://twitter.com/eltibouron
> >
>
-- 

Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message