brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Duncan Grant <duncan.gr...@cloudsoftcorp.com>
Subject Re: YAML blueprint: install package utility
Date Tue, 24 May 2016 08:34:28 GMT
Andrew,

good point.  That's much neater.  Should be pretty straightforward to
implement as well.

Duncan

On Mon, 23 May 2016 at 21:16 Andrew Kennedy <
andrew.kennedy@cloudsoftcorp.com> wrote:

> Duncan,
>
> I like the idea here, of having some extra object that does the package
> installation, but what about a customizer or initializer instead? You could
> have something like this:
>
> - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess
>   brooklyn.initializers:
>   - type: org.apache.brooklyn.initializer.stock.PackageInstaller
>     packages:
>       yum: curl
>       apt: curl-whatever=2.3.4
>
> I initially thought of a JcloudsLocationCustomizer but that fails when we
> use BYON locations, so I am thinking of something more like a generic
> entity customizer that runs after the entity has been created, and the
> location is available and SSHable. There are lots of other things these
> initializers could do, like setting up users, configuring networking,
> mounting volumes and so on. I don't think its a big stretch to add this
> functionality here.
>
> Andrew.
>
> On Fri, 6 May 2016 at 14:40 Duncan Grant <duncan.grant@cloudsoftcorp.com>
> wrote:
>
> > I agree that this is a problem that needs solved but I have misgivings
> > about each of the proposed solutions.
> >
> > Firstly I think that "discovering" how to write brooklyn yaml can be
> > difficult.  It can be difficult to find things in the documentation and
> if
> > you don't know to look for them in the first place then your only hope is
> > coming across an example.
> >
> > So if we pre-install a set of bash scripts before running the install
> > scripts I don't see any obvious way for someone using brooklyn to find
> out
> > about those scripts, and when the scripts change I imagine people won't
> > notice the new functionality either.
> > Another downside to using scripts is that it will become more difficult
> to
> > debug script failures.  At the moment I can copy the contents of stdin
> and
> > see the error associated.  If I had to step into scripts in other files
> it
> > could become much harder to find a failure.  In my opinion this ease of
> > reproducibility and seeing the code I am running is one of the advantages
> > of using bash over just using chef/puppet/salt/etc.
> > If we could find a widely used, well tested bash library then some of
> these
> > issues might be mitigated but I've yet to find one.
> >
> > There is a similar issue with discovering the behaviour of the DSL
> option.
> > I think that the DSL only works because we keep it to a minimum and
> really
> > it only does one task - which is run-time configuring relationships
> between
> > entities.  If we extend it then we have replaced java with dsl and I
> don't
> > think that benefits anyone.  One advantage of the DSL option is that it
> > will generate bash so it shouldn't make debugging any harder.
> >
> > The advantage of adding config options to the VanillaSoftwareProcess is
> > that it adds to extra ways of discovering the behaviour - through the
> > composer's autocomplete and in javadoc.  I assume that this would
> generate
> > extra tasks so we could again debug the bash used and it might make it
> > easier as we might be able to see exactly which package failed to
> install.
> > However the big downside to this is that we are starting to make a sort
> of
> > god object which contains all brooklyn functionality in one object - i.e.
> > VanilllaSoftwareProcess and I'm not sure that's where we want to go.
> >
> > After all that I don't really have a good answer.
> >
> > My not too good answer would be to make more use of the brooklyn child
> > relationship so that a) at least the yaml could be composed from small
> > pieces of code b) we could write a InstallPackage entity to simplify the
> > yaml.
> >
> > This might look something like:
> >
> > - type: brooklyn.entity.basic.VanillaSoftwareProcess
> >   install.command: curl somefile > somfile.txt
> >   brooklyn.config:
> >     children.startable.mode: foreground_early
> >   brooklyn.children:
> >   - type: brooklyn.entity.basic.VanillaSoftwareProcess
> >     install.command: |
> >         which curl || \
> >             { sudo apt-get update && sudo apt-get install curl ; } || \
> >             { sudo yum update && sudo yum install curl ; } || \
> >             { echo WARNING: cannot install curl && exit 1 ; }
> > or
> >   - type: brooklyn.entity.basic.InstallPackage
> >     package: curl
> > or
> >   - type: brooklyn.entity.basic.InstallPackage
> >     packages:
> >       yum: curl
> >       apt: someOtherCurl
> >
> > This way everything is discoverable through documentation, javadoc, and
> > autocomplete.  Common requirements like curl are re-usable. Generated
> bash
> > is explicit and if curl fails then it will go on fire.
> > On the other hand it requires longer yaml, isn't as readable, and due to
> > the use of child parent relationships has a bit of a learning hurdle
> >
> > Regards
> >
> > Duncan
> >
> >
> > On Wed, 4 May 2016 at 17:35 Thomas Bouron <
> thomas.bouron@cloudsoftcorp.com
> > >
> > wrote:
> >
> > > Hi Guglielmo.
> > >
> > > I'm not sure to follow your idea of "stacks", could you elaborate on
> that
> > > please?
> > >
> > > Best.
> > >
> > > On Tue, 3 May 2016, 13:38 Guglielmo Nigri, <
> > > guglielmo.nigri@cloudsoftcorp.com> wrote:
> > >
> > > > I propose a declarative approach. For example we could add a
> configKey
> > to
> > > > VanillaSoftwareProcess called requiredPackages.
> > > >
> > > > This way, one could just specify the prerequisite packages, sort of
> > > > dependencies for the software process -- this would also open up
> > > > interesting possibilities such as “stacks” as bundled dependencies.
> > > >
> > > > A declarative approach would also work for Windows (or non-bash
> > > > environments).
> > > >
> > > > Cheers,
> > > > Guglielmo
> > > >
> > > >
> > > > On 3 May 2016 at 14:28, Andrea Turli <andrea.turli@cloudsoftcorp.com
> >
> > > > wrote:
> > > >
> > > > > Great discussion guys!
> > > > >
> > > > > It seems that it is a shared need, and I wanted to discuss with you
> > as
> > > I
> > > > > was not sure about my approach.
> > > > >
> > > > > Andrew,
> > > > >
> > > > > I like your proposal, thanks!
> > > > >
> > > > > Thanks,
> > > > > Andrea
> > > > >
> > > > >
> > > > > On 3 May 2016 at 13:05, Aled Sage <aled.sage@gmail.com> wrote:
> > > > >
> > > > > > I lean towards Andrew's approach, rather than a special
> > > > > > $brooklyn.installPackage. Note that different distros use
> different
> > > > > package
> > > > > > names sometimes, so the parameters to such a function can get
> > > annoying.
> > > > > >
> > > > > > I've been hesitant about us going down the road of
> > > > `brooklyn-commands.sh`
> > > > > > for achieving portable blueprints. It feels like we are
> increasing
> > > the
> > > > > > overlap with things like Chef, Salt, Ansible and Puppet (which
we
> > can
> > > > > build
> > > > > > on top of with Brooklyn).
> > > > > >
> > > > > > However, if we keep brooklyn-commands.sh small and focused,
then
> > > maybe
> > > > > > it's a good idea.
> > > > > >
> > > > > > Aled
> > > > > >
> > > > > > p.s. I want to decrease the use of
> > > `BashCommands.installExecutable()`.
> > > > > The
> > > > > > bash command it generates, with all the alternatives and
> > parentheses,
> > > > > does
> > > > > > not look like nice Bash. A brooklyn-commands.sh could do it
much
> > > > cleaner,
> > > > > > with a properly structured multi-line if...elif... (or whatever).
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 03/05/2016 11:43, Andrew Kennedy wrote:
> > > > > >
> > > > > >> How about an alternative approach - for all SoftwareProcess
> > > entities,
> > > > > >> Brooklyn will copy over a script file called
> > `brooklyn-commands.sh`
> > > > and
> > > > > >> the
> > > > > >> entity commands will soutce this script before running the
rest
> of
> > > > their
> > > > > >> configured commands. The script will contain OS agnostic
> functions
> > > > > written
> > > > > >> in Bash that do things like install packages, download Curl,
> get a
> > > > file
> > > > > >> from a URL etc. Then, the install config might look like
this:
> > > > > >>
> > > > > >> ```
> > > > > >> install.commands: |
> > > > > >>    brooklyn-installPackages apt:openjdk-1.8.0
> > > > > yum:java-1.8.0-openjdk-devel
> > > > > >> java-1.8.0
> > > > > >>    brooklyn-installPackages curl
> > > > > >>    brooklyn-runAsRoot cp /tmp/whatever /etc/hosts
> > > > > >> ```
> > > > > >>
> > > > > >> Andrew.
> > > > > >>
> > > > > >> On Tue, 3 May 2016 at 11:24 Andrea Turli <
> > > > > andrea.turli@cloudsoftcorp.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> hi,
> > > > > >>>
> > > > > >>> I’ve been thinking about an utility to simplify the
YAML
> > blueprint
> > > > > >>> creation: from my experience when using VanillaSoftwareProcess
> is
> > > > > >>> annoying
> > > > > >>> to write a portable script just to install a package
(say,
> java)
> > > > valid
> > > > > >>> for
> > > > > >>> apt, yum, etc so I usually write it (multiple times)
just for
> an
> > > OS.
> > > > > >>>
> > > > > >>> To increase the portability of the YAML blueprint I’d
like to
> > > suggest
> > > > > we
> > > > > >>> extend the brooklyn DSL with something like:
> > > > > >>> ```
> > > > > >>> $brooklyn.installPackage(“curl”)
> > > > > >>> $brooklyn:installPackage({"apt", "openjdk-1.8.0", "yum",
> > > > > >>> "java-1.8.0-openjdk-devel"}, "java-1.8.0")
> > > > > >>> ```
> > > > > >>> instead of things like
> > > > > >>> ```
> > > > > >>> which curl || \
> > > > > >>>          { sudo apt-get update && sudo apt-get
install curl ; }
> > ||
> > > \
> > > > > >>>          { sudo yum update && sudo yum install
curl ; } || \
> > > > > >>>          { echo WARNING: cannot install curl &&
exit 1 ; }
> > > > > >>> ```
> > > > > >>> I’m not entirely sure this feature fits well on the
DSL.
> > > > > >>>
> > > > > >>> Alternatively, we could add a configKey to
> VanillaSoftwareProcess
> > > > > called
> > > > > >>> requiredPackages for a more declarative approach (@googlielmo's
> > > idea)
> > > > > >>>
> > > > > >>> Wdyt?
> > > > > >>>
> > > > > >>>
> > > > > >
> > > > >
> > > >
> > > --
> > > Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
> > > http://www.cloudsoftcorp.com/
> > > Github: https://github.com/tbouron
> > > Twitter: https://twitter.com/eltibouron
> > >
> >
> --
>
> Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message