hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Hadoop build slaves software
Date Mon, 07 Jan 2013 23:45:14 GMT
OK. FYI, installed protoc on asf009, and the "g++-4.4-multilib"
packages on both asf008 and asf009. Checked the hadoop pipes native
build and it passes now. Fingers crossed...

-Todd

On Mon, Jan 7, 2013 at 3:35 PM, Todd Lipcon <todd@cloudera.com> wrote:
> I'll install the right protoc and libstdc++ dev on asf009 as well.
>
> -Todd
>
> On Mon, Jan 7, 2013 at 9:57 AM, Andrew Wang <andrew.wang@cloudera.com> wrote:
>> I think hadoop9 has a similar problem as hadoop8, based on a recent build.
>> The javac output has a compile-proto error:
>>
>> https://builds.apache.org/job/PreCommit-HDFS-Build/3755/
>> https://builds.apache.org/job/PreCommit-HDFS-Build/3755/artifact/trunk/patchprocess/trunkJavacWarnings.txt
>>
>>
>> On Sun, Jan 6, 2013 at 1:57 AM, Binglin Chang <decstery@gmail.com> wrote:
>>
>>> HAServiceProtocol.proto:21:8: Option "java_generic_services" unknown.
>>> This is probably caused by a older version of protoc in build env.
>>>
>>>
>>> On Sun, Jan 6, 2013 at 2:12 PM, Giridharan Kesavan <
>>> gkesavan@hortonworks.com
>>> > wrote:
>>>
>>> > by looking at the failure log :
>>> >
>>> >
>>> https://builds.apache.org/view/Hadoop/job/PreCommit-HADOOP-Build/1950/artifact/trunk/patchprocess/trunkJavacWarnings.txt
>>> > build failed on
>>> >
>>> > [INFO] --- exec-maven-plugin:1.2:exec (compile-proto) @ hadoop-common ---
>>> >
>>> > HAServiceProtocol.proto:21:8: Option "java_generic_services" unknown.
>>> >
>>> > I'm not sure if this is something to do with the build env.
>>> >
>>> > -Giri
>>> >
>>> >
>>> > On Sat, Jan 5, 2013 at 5:57 PM, Binglin Chang <decstery@gmail.com>
>>> wrote:
>>> >
>>> > > I am not sure if this problem is solved, the build still failed in
>>> > > precommit-HADOOP
>>> > > https://builds.apache.org/view/Hadoop/job/PreCommit-HADOOP-Build/
>>> > >
>>> > >
>>> > > On Sat, Jan 5, 2013 at 6:46 AM, Giridharan Kesavan <
>>> > > gkesavan@hortonworks.com
>>> > > > wrote:
>>> > >
>>> > > > Marking the slave offline would do. I 've mared the hadoop8 slave
>>> > > offline,
>>> > > > while I test it for builds and bring it back online later when
its
>>> > good.
>>> > > >
>>> > > >
>>> > > > -Giri
>>> > > >
>>> > > >
>>> > > > On Fri, Jan 4, 2013 at 2:26 PM, Todd Lipcon <todd@cloudera.com>
>>> wrote:
>>> > > >
>>> > > > > Turns out I had to both kill -9 it and chmod 000
>>> > > > > /home/jenkins/jenkins-slave in order to keep it from
>>> auto-respawning.
>>> > > > > Just a note so that once the toolchain is fixed, someone
knows to
>>> > > > > re-chmod back to 755.
>>> > > > >
>>> > > > > -Todd
>>> > > > >
>>> > > > > On Fri, Jan 4, 2013 at 2:11 PM, Todd Lipcon <todd@cloudera.com>
>>> > wrote:
>>> > > > > > I'm going to kill -9 the jenkins slave on hadoop8 for
now cuz
>>> it's
>>> > > > > > causing havoc on the precommit builds. I can't see another
way to
>>> > > > > > administratively disable it from the Jenkins interface.
>>> > > > > >
>>> > > > > > Rajiv, Giri -- mind if I build/install protoc into /usr/local
to
>>> > > match
>>> > > > > > the other slaves? We can continue the conversation about
>>> > provisioning
>>> > > > > > after, but would like to unblock the builds in the meantime.
>>> > > > > >
>>> > > > > > As for CentOS vs Ubuntu, I've got no preference. RHEL6
is
>>> probably
>>> > > > > > preferable since it's a more common install platform,
anyway.
>>> But,
>>> > > > > > we'll still need to have a custom toolchain for things
like
>>> protoc
>>> > > 2.4
>>> > > > > > which don't have new enough versions in the package
repos.
>>> > > > > >
>>> > > > > > -Todd
>>> > > > > >
>>> > > > > > On Fri, Jan 4, 2013 at 2:03 PM, Colin McCabe <
>>> > cmccabe@alumni.cmu.edu
>>> > > >
>>> > > > > wrote:
>>> > > > > >> In addition to protoc, can someone please also install
a 32-bit
>>> > C++
>>> > > > > compiler?
>>> > > > > >>
>>> > > > > >> The builds are all failing on this machine because
of that.
>>> > > > > >>
>>> > > > > >> regards,
>>> > > > > >> Colin
>>> > > > > >>
>>> > > > > >>
>>> > > > > >> On Fri, Jan 4, 2013 at 11:37 AM, Giridharan Kesavan
>>> > > > > >> <gkesavan@hortonworks.com> wrote:
>>> > > > > >>> When I configured the other machines I used
the source to
>>> compile
>>> > > and
>>> > > > > >>> install the protoc, as the 2.4.1 wasn't available
in the ubuntu
>>> > > repo.
>>> > > > > >>>
>>> > > > > >>> BTW installed 2.4.1 on asf008.
>>> > > > > >>> gkesavan@asf008:~$ protoc --version
>>> > > > > >>> libprotoc 2.4.1
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>> -Giri
>>> > > > > >>>
>>> > > > > >>>
>>> > > > > >>> On Thu, Jan 3, 2013 at 11:24 PM, Todd Lipcon
<
>>> todd@cloudera.com>
>>> > > > > wrote:
>>> > > > > >>>
>>> > > > > >>>> Hey folks,
>>> > > > > >>>>
>>> > > > > >>>> It looks like hadoop8 has recently come
back online as a build
>>> > > > slave,
>>> > > > > >>>> but is failing all the builds because it
has an ancient
>>> version
>>> > of
>>> > > > > >>>> protobuf (2.2.0):
>>> > > > > >>>> todd@asf008:~$ protoc  --version
>>> > > > > >>>> libprotoc 2.2.0
>>> > > > > >>>>
>>> > > > > >>>> In contrast, other slaves have 2.4.1:
>>> > > > > >>>> todd@asf001:~$ protoc --version
>>> > > > > >>>> libprotoc 2.4.1
>>> > > > > >>>>
>>> > > > > >>>> asf001 has the newer protoc in /usr/local/bin
but asf008 does
>>> > not.
>>> > > > > >>>> Does anyone know how software is meant to
be deployed on these
>>> > > build
>>> > > > > >>>> slaves? I'm happy to download and install
protobuf 2.4.1 into
>>> > > > > >>>> /usr/local on asf008 if manual installation
is the name of the
>>> > > game,
>>> > > > > >>>> but it seems like we should be doing something
a little more
>>> > > > > >>>> reproducible than one-off builds by rando
developers to manage
>>> > our
>>> > > > > >>>> toolchain on the Jenkins slaves.
>>> > > > > >>>>
>>> > > > > >>>> -Todd
>>> > > > > >>>> --
>>> > > > > >>>> Todd Lipcon
>>> > > > > >>>> Software Engineer, Cloudera
>>> > > > > >>>>
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > --
>>> > > > > > Todd Lipcon
>>> > > > > > Software Engineer, Cloudera
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Todd Lipcon
>>> > > > > Software Engineer, Cloudera
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message