jclouds-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ignasi Barrera <n...@apache.org>
Subject Re: Problem bootstraping FreeBSD on EC2
Date Tue, 02 Feb 2016 23:44:37 GMT
Great investigation, and thanks for reporting back the results!

The bootstrap BSD looks like a nice recipe to have. I'd add a statement
class for it to the preconfigured Statements [1] so users can easily use
that when composing their scripts. And the ComputeService could also add
that automatically to BSD nodes when needed.

Also, being able to add a user to the sudores file would be a nice addition
to the SudoStatements [2].

Mind opening a JIRA to track this? And even better, are you up to
contributing the changes in a pull request? :)

Thanks!

I.

[1]
https://github.com/jclouds/jclouds/tree/master/scriptbuilder/src/main/java/org/jclouds/scriptbuilder/statements
[2]
https://github.com/jclouds/jclouds/blob/master/scriptbuilder/src/main/java/org/jclouds/scriptbuilder/statements/login/SudoStatements.java
El 3/2/2016 12:07 a. m., "Klemen Ferjančič" <imbacen@gmail.com> escribió:

> The problem is that FreeBSD /bin/sh really does not like DOS style line
> breaks so it interprets \r as part of the command name, therefore the
> weird "not found" messages. I ended up with replacing \r\n with \n in
> java on the fly, then uploading script to the server and executing it
> with a one liner:
>
> su - root -c "chmod +x /home/ec2-user/bootstrap_freebsd.sh &&
> /home/ec2-user/bootstrap_freebsd.sh"
>
> bootstrap_freebsd.sh content:
> #!/bin/sh
> pkg install -y sudo
> pkg install -y bash
> ln /usr/local/bin/bash /bin/bash
> echo "ec2-user ALL=(ALL) NOPASSWD:ALL" >> /usr/local/etc/sudoers
>
>
> You need to install sudo, bash, add ec2-user to sudoers and also symlink
> the bash path because jclouds uses /bin/bash hardcoded in the init files.
>
> After these 4 lines are executed you can proceed executing commands with
> init script and sudo as normal.
>
> Hopefully this helps someone in the future as crazy as I who wants to
> run FreeBSD. :)
>
> Best regards, Klemen
>
>
> On 29. 01. 2016 14:55, Ignasi Barrera wrote:
> > That one-liner thing is really weird! Glad to see you found a way to
> > bypass that.
> >
> > Another option would be to put your script in a file, upload it to the
> > node using the jclouds ssh client, and then run a script that simply
> > executes that script file. Could that work and be cleaner? Have a look
> > at the "Advanced usage" section of the compute guide [1] for an
> > example of how to upload a file to a node.
> >
> > I.
> >
> > [1] http://jclouds.apache.org/start/compute/
> >
> > On 28 January 2016 at 23:14, cen <imbacen@gmail.com> wrote:
> >> Found the solution. If I execute commands in a single line using &&,
> written
> >> as a java string they work as expected.
> >> But reading them from a file where I write them line by line it cause
> the
> >> problems mentioned previously:
> >>
> >> Files.toString(file, Charsets.UTF_8)
> >>
> >> If I write them in a single line in the file it works as expected too. I
> >> tried to print the commands when they are line by line
> >> and everything seems normal in console. Maybe it's the line breaks in
> >> combination with tcsh or something in that regard causing the problem.
> >>
> >> Plus another find: you can't actually su root && command, the proper
> way is
> >> su - root -c "commands && here". su root seems to break the flow.
> >>
> >>
> >> Ignasi Barrera je 28. 01. 2016 ob 11:12 napisal:
> >>>
> >>> I have no experience with BSD instances, but could it be a PATH issue?
> >>>
> >>> Can you try setting the PATH manually?
> >>>
> >>> On 28 January 2016 at 09:13, Klemen Ferjančič <imbacen@gmail.com>
> wrote:
> >>>>
> >>>> It seems I've hit a wall.
> >>>>
> >>>> 1. For some reason, even when disabling init script on template
> builder,
> >>>> it still tries to run it. So I decided to skip it for now.
> >>>>
> >>>>
> >>>>
> .wrapInInitScript(false).runScript(getBootInstructions(os)).runAsRoot(true);
> >>>>
> >>>> 2. Running as root and disabling sudo does not actually remove sudo.
> >>>> Taking a closer look at RunScriptOnNodeUsingSsh.java->execAsRoot,
it
> >>>> seems that sudo is always run in front.
> >>>>
> >>>>
> >>>>
> opts.wrapInInitScript(false).runAsRoot(true).overrideAuthenticateSudo(false)
> >>>> .overrideLoginCredentials(getLoginForCommandExecution(os));
> >>>>
> >>>> Result:
> >>>> `sudo sh <<'RUN_SCRIPT_AS_ROOT_SSH' ...
> >>>>
> >>>> which means I can't run as root.
> >>>>
> >>>> 3. So I decided to run as regular user (not root and not sudo) and
> just
> >>>> switch to root manually. Works if I do it by hand:
> >>>> su root
> >>>> pkg install -y sudo
> >>>> pkg install -y bash
> >>>>
> >>>> Opts:
> >>>> .wrapInInitScript(false).runAsRoot(false)
> >>>> .overrideLoginCredentials(getLoginForCommandExecution(os))
> >>>>
> >>>> But for some reason, it seems that jclouds puts me in some weird
> >>>> restricted shell where I can't execute anything.
> >>>> su returns "Sorry" and trying to run a simple command like "whoami"
> >>>> results in: not found. It's clear to me that this is not the same
> >>>> environment as the one
> >>>> when I manually ssh into the machine so what is the difference?
> >>>>
> >>>>
> >>>> Best regards, Klemen
> >>>>
> >>>> On 27. 01. 2016 11:06, Ignasi Barrera wrote:
> >>>>>
> >>>>> No. You can use the one in the template builder to run the first
one
> and
> >>>>> the run the rest by calling computeService.runScriptOnNode
> afterhaving
> >>>>> created the node.
> >>>>>
> >>>>> El 27/1/2016 11:03 a. m., "Klemen Ferjančič" <imbacen@gmail.com
> >>>>> <mailto:imbacen@gmail.com>> escribió:
> >>>>>
> >>>>>      If I add additional scripts on top of template builder are
they
> >>>>> going to
> >>>>>      be executed in order or is the solution more complex than this?
> >>>>>
> >>>>>      Something like:
> >>>>>
> >>>>>
> >>>>>
> templateBuilder.wrapInInitScript(false).runScript(installBashAndSudo())
> >>>>>      .wrapInitScript(true).runScript(everythingElse())
> >>>>>
> >>>>>      On 27. 01. 2016 10:56, Ignasi Barrera wrote:
> >>>>>      > Yes, the RunScriptOptions provide options to run the scripts
> as
> >>>>> root,
> >>>>>      > and also to explicitly disable sudo.
> >>>>>      >
> >>>>>      > Currently the wrapper script requires bash, so you'll
have to
> >>>>> install
> >>>>>      > it first if you need to use it. You could split the runscript
> in
> >>>>> two
> >>>>>      > operations: a first one with the wrapInitScript(false)
to just
> >>>>> install
> >>>>>      > bash, and a second one to run the desired script.
> >>>>>      >
> >>>>>      > That wrapper script, as said, contains several helpers
to let
> >>>>> jclouds
> >>>>>      > query the status of the script execution: see if it is
still
> >>>>> running,
> >>>>>      > allow to abort it, etc. If you are running scripts that
take
> >>>>> time, I'd
> >>>>>      > recommend you run them with the wrapper script.
> >>>>>      >
> >>>>>      > I.
> >>>>>      >
> >>>>>      > On 27 January 2016 at 10:48, Klemen Ferjančič <
> imbacen@gmail.com
> >>>>>      <mailto:imbacen@gmail.com>> wrote:
> >>>>>      >> Good advice, I suspect I know what is going on.
> >>>>>      >>
> >>>>>      >> Home directory contains bootstrap and /tmp/init-bootstrap
> >>>>> exists.
> >>>>>      >> However, if I run ./init-bootstrap it says "not found".
I
> >>>>> checked the
> >>>>>      >> init script and it seems that bash is expected to
exist on
> the
> >>>>> system
> >>>>>      >> (#!/bin/bash), however, bash does not come preinstalled
on
> >>>>> FreeBSD. I
> >>>>>      >> could install bash with runScript but I don't think
personal
> >>>>>      script is
> >>>>>      >> executed before the init one? Another interesting
note: sudo
> is
> >>>>> not
> >>>>>      >> preinstalled either so I probably need to run my script
as
> root.
> >>>>> I'll
> >>>>>      >> play around with this and see what I can do.
> >>>>>      >>
> >>>>>      >> Code snippet:
> >>>>>      >>
> >>>>>      >> templateBuilder.osFamily(OsFamily.FREEBSD);
> >>>>>      >> templateBuilder.imageId(REGION + "/ami-9f5549f3");
> >>>>>      >> EC2TemplateOptions o =
> >>>>>      EC2TemplateOptions.Builder.keyPair("mykeypair")
> >>>>>      >>
> >>>>>
> >>>>>
> .overrideLoginCredentials(getLoginForCommandExecution(os)).runScript(getBootInstructions(os));
> >>>>>      >> templateBuilder.locationId(REGION);
> >>>>>      >> templateBuilder.hardwareId(INSTANCE);
> >>>>>      >> templateBuilder.options(o);
> >>>>>      >>
> >>>>>      >>
> >>>>>      >> On 27. 01. 2016 10:23, Ignasi Barrera wrote:
> >>>>>      >>> Hi Klemen,
> >>>>>      >>>
> >>>>>      >>> jclouds generates and uploads that script to the
node. It
> is a
> >>>>>      wrapper
> >>>>>      >>> script that provides some helpers to get the status
of the
> >>>>>      configured
> >>>>>      >>> script, and to properly capture the output, stderr,
and
> make it
> >>>>>      >>> possible to abort its execution.
> >>>>>      >>>
> >>>>>      >>> After the failure, can you log in to the instance
and see
> which
> >>>>>      files
> >>>>>      >>> do you have in the user's home directory and in
/tmp?
> >>>>>      >>>
> >>>>>      >>> You can also disable the wrapper script by configuring
the
> >>>>>      >>> "wrapInitScript(false)" in the RunScriptOptions,
and see if
> the
> >>>>>      >>> problem persists.
> >>>>>      >>>
> >>>>>      >>> Could you also share a small code snippet of the
code you're
> >>>>>      using to
> >>>>>      >>> bootstrap the node?
> >>>>>      >>>
> >>>>>      >>> I.
> >>>>>      >>>
> >>>>>      >>> On 27 January 2016 at 09:39, Klemen Ferjančič
> >>>>> <imbacen@gmail.com
> >>>>>      <mailto:imbacen@gmail.com>> wrote:
> >>>>>      >>>> Hi
> >>>>>      >>>>
> >>>>>      >>>> When I create a new Ec2 instance with FreeBSD
private AMI,
> the
> >>>>>      >>>> init-bootstrap fails to run. The instance
is created and
> runs
> >>>>>      normally,
> >>>>>      >>>> but the error also fails my own runScript
so I'd like to
> >>>>>      resolve it.
> >>>>>      >>>>
> >>>>>      >>>> 00:32:27,588 ERROR [jclouds.compute] (user
thread 0) <<
> >>>>> problem
> >>>>>      >>>> customizing node(eu-central-1/i-22cd4c9e):
:
> >>>>>      >>>> java.lang.IllegalStateException: error running
> >>>>> [/tmp/init-bootstrap
> >>>>>      >>>> init] as ec2-user@52.59.247.117
> >>>>>      <mailto:ec2-user@52.59.247.117>; returnVal !=0:
> >>>>>      >>>> {output=/tmp/init-bootstrap: not found
> >>>>>      >>>> , error=, exitStatus=127)
> >>>>>      >>>>
> >>>>>      >>>> Full stacktrace: http://pastebin.com/xabqprs5
> >>>>>      >>>>
> >>>>>      >>>> 1. Is /tmp/init-bootstrap supposed to already
exist on the
> >>>>>      image or is
> >>>>>      >>>> this a script that jclouds uploads to the
node? If the
> latter,
> >>>>>      I assume
> >>>>>      >>>> there is a problem with script not being uploaded
in the
> first
> >>>>>      place? No
> >>>>>      >>>> error indicates this though.
> >>>>>      >>>> 2. What does this script actually do? Is it
possible to
> >>>>> disable it?
> >>>>>      >>>>
> >>>>>      >>>> Best regards, Klemen
> >>>>>      >>>>
> >>>>>      >>>>
> >>>>>      >>
> >>>>>
> >>
>
>

Mime
View raw message