hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szehon Ho <sze...@cloudera.com>
Subject Re: Plan: permanently move hive builds from bigtop
Date Tue, 22 Apr 2014 23:15:22 GMT
Patch upload auto-trigger is now enabled, thanks Ashutosh for helping me
fix the Jenkins url from Apache-Infra side.  Now there's no need to
manually submit Pre-commit builds, unless it got missed somehow.  And in
this Jenkins, you can easily see all the JIRA's in the queue:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/

Sorry, please use this URL and not the first one, as job was renamed due to
the auto-trigger expecting this format.

And to repeat, Test Reporting and the 'Logs located at:' link is still
broken and tracked in HIVE-6937.  For now, to see the logs of a failed
build, you have to manually navigate.
Example, for PreCommit-Hive-BUILD#6, logs are at:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-6/<http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-6/>.
The HiveQA comment on the JIRA should mention the build number.

Thanks,
Szehon


On Sun, Apr 20, 2014 at 11:07 PM, Szehon Ho <szehon@cloudera.com> wrote:

> It looks great, thanks Lefty!
>
>
> On Sun, Apr 20, 2014 at 2:22 PM, Lefty Leverenz <leftyleverenz@gmail.com>wrote:
>
>> Nice doc, Szehon.  I did some minor editing so you might want to make sure
>> I didn't introduce any errors.
>>
>>
>> https://cwiki.apache.org/confluence/display/Hive/Hive+PTest2+Infrastructure
>>
>> -- Lefty
>>
>>
>> On Sat, Apr 19, 2014 at 9:45 PM, Szehon Ho <szehon@cloudera.com> wrote:
>>
>> > Migration is done, I updated the wiki to add all the details of the new
>> > setup:
>> >
>> https://cwiki.apache.org/confluence/display/Hive/Hive+PTest2+Infrastructure
>> >
>> > New Jenkins URL to submit pre-commit jobs:
>> >
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/precommit-hive/
>> >   Again, this has to be done manually for time being, by clicking on
>> 'build
>> > with parameters', and entering the issue number as a parameter.  I've
>> > submitted some already.  I'll reach out to some committers to get the
>> > auto-trigger working.
>> >
>> > As I mentioned, there is some work to fix the test-reporting, due to the
>> > framework using old url scheme.  I am tracking it at
>> > HIVE-6937<https://issues.apache.org/jira/browse/HIVE-6937>.
>> >  For now I am hosting log directory separately, if you want to see test
>> > logs, you have to manually go the url corresponding to your build, like:
>> >
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/precommit-hive-11/for
>> > run#11.  Sorry about that.
>> >
>> > Let me know if you see other issues, thanks!
>> > Szehon
>> >
>> >
>> > On Fri, Apr 18, 2014 at 2:11 PM, Thejas Nair <thejas@hortonworks.com>
>> > wrote:
>> >
>> > > Sounds good.
>> > > Thanks Szehon!
>> > >
>> > >
>> > > On Fri, Apr 18, 2014 at 10:17 AM, Ashutosh Chauhan <
>> hashutosh@apache.org
>> > >
>> > > wrote:
>> > > > +1 Thanks Szehon!
>> > > >
>> > > >
>> > > > On Fri, Apr 18, 2014 at 6:29 AM, Xuefu Zhang <xzhang@cloudera.com>
>> > > wrote:
>> > > >
>> > > >> +1. Thanks for taking care of this.
>> > > >>
>> > > >>
>> > > >> On Thu, Apr 17, 2014 at 11:00 PM, Szehon Ho <szehon@cloudera.com>
>> > > wrote:
>> > > >>
>> > > >> > Hi,
>> > > >> >
>> > > >> > This week the machine running Hive builds at
>> > > >> > http://bigtop01.cloudera.org:8080/view/Hive/?  ran out of
>> space, so
>> > > new
>> > > >> > jobs like Precommit tests stopped.  Its still not resolved
there,
>> > > there
>> > > >> was
>> > > >> > another email today on Bigtop list, but there's very few
people
>> with
>> > > root
>> > > >> > access to that host, and they still haven't responded.
>> > > >> >
>> > > >> > I chatted with Brock, he has also seen various issues with
Bigtop
>> > > jenkins
>> > > >> > in the past, so I am thinking to move the Jenkins jobs to
the
>> PTest
>> > > >> master
>> > > >> > itself, where some PMC already have access and can admin
if
>> needed.
>> > > >> >  Currently I am hosting the pre-commit Jenkins job on my
own EC2
>> > > instance
>> > > >> > as stop-gap.
>> > > >> >
>> > > >> > Other advantages of hosting our own Jenkins:
>> > > >> > 1. No need to wait for other Bigtop jobs to run.
>> > > >> > 2. Bigtop is using a version of Jenkins that doesnt show
>> parameters
>> > > like
>> > > >> > JIRA number for queued jobs, so impossible to tell whether
a
>> patch
>> > got
>> > > >> > picked up and where it is in queue.
>> > > >> > 3. Eliminate network hop from Bigtop box to our PTest master.
>> > > >> >
>> > > >> > The disadvantage is:
>> > > >> > 1. We don't have much experience doing Jenkins admin, but
it
>> doesn't
>> > > look
>> > > >> > too bad.  Mostly, restart if there's issue and clean up if
out of
>> > > space.
>> > > >> >
>> > > >> > I wonder what people think, and if there's any objections
to
>> this?
>> >  If
>> > > >> not,
>> > > >> > I'll try setting up this weekend.  Then, there is some follow-up
>> > work,
>> > > >> like
>> > > >> > changing the Jenkins url's displayed in the test report.
>> > > >> >
>> > > >> > Thanks!
>> > > >> > Szehon
>> > > >> >
>> > > >>
>> > >
>> > > --
>> > > CONFIDENTIALITY NOTICE
>> > > NOTICE: This message is intended for the use of the individual or
>> entity
>> > to
>> > > which it is addressed and may contain information that is
>> confidential,
>> > > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > > of this message is not the intended recipient, you are hereby notified
>> > that
>> > > any printing, copying, dissemination, distribution, disclosure or
>> > > forwarding of this communication is strictly prohibited. If you have
>> > > received this communication in error, please contact the sender
>> > immediately
>> > > and delete it from your system. Thank You.
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message