impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Apple <jbap...@cloudera.com>
Subject Re: GVO machines dependencies
Date Thu, 25 May 2017 16:18:37 GMT
Nice find, Sailesh!

On Wed, May 24, 2017 at 6:39 PM, Sailesh Mukil <sailesh@cloudera.com> wrote:
> The issue was that there was a bug in the install.sh script from the
> impala-setup/ repo that always assumed that the repo would reside in
> /home/$user.
>
> The Jenkins AMIs seem to have had a very old checkout (8 months old) at
> /home/ubuntu/ which was being used for the GVOs. The Jenkins job would make
> a fresh checkout in /tmp/ and run install.sh from that checkout. But due to
> the above bug, the script would change its sources to the old repo in
> /home/ubuntu. So, the new dependencies added to the project didn't get
> picked up. I've submitted a fix, and will redo the GVO once it's merged
> into impala-setup.
>
> Thanks for your help Michael and Jim!
>
> On Wed, May 24, 2017 at 4:58 PM, Michael Brown <mikeb@cloudera.com> wrote:
>
>> "from-scratch" is a misnomer: only the first build to run on a given node
>> is truly from-scratch, because running impala-setup takes effect
>> system-wide, and is run once, when the node is created.
>>
>> For http://jenkins.impala.io:8080/job/ubuntu-14.04-from-scratch/1371/ ,
>> the
>> worker's build history suggests it has been up roughly 17 hours.
>> http://jenkins.impala.io:8080/computer/ub1404-c4.4xl-gp2%20(
>> i-0d76efbc8de26926c)/builds
>> This means your recent change hasn't taken effect on "older" workers.
>>
>> Any new workers that get created should get the change.
>>
>>
>>
>>
>>
>> On Wed, May 24, 2017 at 4:34 PM, Sailesh Mukil <sailesh@cloudera.com>
>> wrote:
>>
>> > Thanks Michael and Jim,
>> >
>> > It looks like adding to bin/bootstrap_build.sh makes the ub14-build-only
>> > job work fine, however, updating the impala-setup still doesn't seem to
>> > work. I submitted a pull request that was merged by Dimitris here:
>> > https://github.com/awleblang/impala-setup/commits/master
>> >
>> > Also, the impala-setup chef logs don't seem to get printed in the jenkins
>> > console output. So I'm not sure if it's actually being run or not.
>> >
>> > On Wed, May 24, 2017 at 3:37 PM, Michael Brown <mikeb@cloudera.com>
>> wrote:
>> >
>> > > Thanks Jim; I had forgotten bin/bootstrap_build.sh.
>> > >
>> > > On Wed, May 24, 2017 at 3:14 PM, Jim Apple <jbapple@cloudera.com>
>> wrote:
>> > >
>> > > > I'd recommend against #2 using the node setup to install Impala
>> > > > dependencies. The only reason it installs openjdk is that Jenkins
>> > > > needs Java to talk to it. All of the other Impala dependencies are
>> > > > installed in the jobs themselves.
>> > > >
>> > > > On Wed, May 24, 2017 at 3:11 PM, Michael Brown <mikeb@cloudera.com>
>> > > wrote:
>> > > > >> Is there a way to add a new dependency to these machines?
>> > > > >
>> > > > > Our builds use two worker labels:
>> > > > > 1. ubuntu14.04-c4.4xlarge-gp2
>> > > > > 2. ub14-build-only
>> > > > >
>> > > > > 1 uses https://github.com/awleblang/impala-setup, and I think
you
>> > > should
>> > > > > add libffi to there. It's something you should do anyway so new
>> users
>> > > get
>> > > > > set up with the right dependencies.
>> > > > >
>> > > > > 2 is a different story, but I see that at
>> > > > > http://jenkins.impala.io:8080/configure that worker is already
>> > > > installing a
>> > > > > JDK via apt-get. You could use a similar pattern to install libffi.
>> > > >
>> > >
>> >
>>

Mime
View raw message