impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sailesh Mukil <sail...@cloudera.com>
Subject Re: GVO machines dependencies
Date Thu, 25 May 2017 17:06:13 GMT
Glad I could help! The fix is merged now:
https://github.com/awleblang/impala-setup/commit/1fcff44887de76fdb5f86948635c073d0e615f1c

On Thu, May 25, 2017 at 10:00 AM, Laszlo Gaal <laszlo.gaal@cloudera.com>
wrote:

> Thanks a lot for fixing this, Sailesh, this has been bugging me for a while
> when spinning up cloud-based workstations.
>
> On Thu, May 25, 2017 at 6:18 PM, Jim Apple <jbapple@cloudera.com> wrote:
>
> > Nice find, Sailesh!
> >
> > On Wed, May 24, 2017 at 6:39 PM, Sailesh Mukil <sailesh@cloudera.com>
> > wrote:
> > > The issue was that there was a bug in the install.sh script from the
> > > impala-setup/ repo that always assumed that the repo would reside in
> > > /home/$user.
> > >
> > > The Jenkins AMIs seem to have had a very old checkout (8 months old) at
> > > /home/ubuntu/ which was being used for the GVOs. The Jenkins job would
> > make
> > > a fresh checkout in /tmp/ and run install.sh from that checkout. But
> due
> > to
> > > the above bug, the script would change its sources to the old repo in
> > > /home/ubuntu. So, the new dependencies added to the project didn't get
> > > picked up. I've submitted a fix, and will redo the GVO once it's merged
> > > into impala-setup.
> > >
> > > Thanks for your help Michael and Jim!
> > >
> > > On Wed, May 24, 2017 at 4:58 PM, Michael Brown <mikeb@cloudera.com>
> > wrote:
> > >
> > >> "from-scratch" is a misnomer: only the first build to run on a given
> > node
> > >> is truly from-scratch, because running impala-setup takes effect
> > >> system-wide, and is run once, when the node is created.
> > >>
> > >> For http://jenkins.impala.io:8080/job/ubuntu-14.04-from-scratch/1371/
> ,
> > >> the
> > >> worker's build history suggests it has been up roughly 17 hours.
> > >> http://jenkins.impala.io:8080/computer/ub1404-c4.4xl-gp2%20(
> > >> i-0d76efbc8de26926c)/builds
> > >> This means your recent change hasn't taken effect on "older" workers.
> > >>
> > >> Any new workers that get created should get the change.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Wed, May 24, 2017 at 4:34 PM, Sailesh Mukil <sailesh@cloudera.com>
> > >> wrote:
> > >>
> > >> > Thanks Michael and Jim,
> > >> >
> > >> > It looks like adding to bin/bootstrap_build.sh makes the
> > ub14-build-only
> > >> > job work fine, however, updating the impala-setup still doesn't seem
> > to
> > >> > work. I submitted a pull request that was merged by Dimitris here:
> > >> > https://github.com/awleblang/impala-setup/commits/master
> > >> >
> > >> > Also, the impala-setup chef logs don't seem to get printed in the
> > jenkins
> > >> > console output. So I'm not sure if it's actually being run or not.
> > >> >
> > >> > On Wed, May 24, 2017 at 3:37 PM, Michael Brown <mikeb@cloudera.com>
> > >> wrote:
> > >> >
> > >> > > Thanks Jim; I had forgotten bin/bootstrap_build.sh.
> > >> > >
> > >> > > On Wed, May 24, 2017 at 3:14 PM, Jim Apple <jbapple@cloudera.com>
> > >> wrote:
> > >> > >
> > >> > > > I'd recommend against #2 using the node setup to install
Impala
> > >> > > > dependencies. The only reason it installs openjdk is that
> Jenkins
> > >> > > > needs Java to talk to it. All of the other Impala dependencies
> are
> > >> > > > installed in the jobs themselves.
> > >> > > >
> > >> > > > On Wed, May 24, 2017 at 3:11 PM, Michael Brown <
> > mikeb@cloudera.com>
> > >> > > wrote:
> > >> > > > >> Is there a way to add a new dependency to these
machines?
> > >> > > > >
> > >> > > > > Our builds use two worker labels:
> > >> > > > > 1. ubuntu14.04-c4.4xlarge-gp2
> > >> > > > > 2. ub14-build-only
> > >> > > > >
> > >> > > > > 1 uses https://github.com/awleblang/impala-setup, and
I think
> > you
> > >> > > should
> > >> > > > > add libffi to there. It's something you should do anyway
so
> new
> > >> users
> > >> > > get
> > >> > > > > set up with the right dependencies.
> > >> > > > >
> > >> > > > > 2 is a different story, but I see that at
> > >> > > > > http://jenkins.impala.io:8080/configure that worker
is
> already
> > >> > > > installing a
> > >> > > > > JDK via apt-get. You could use a similar pattern to
install
> > libffi.
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message