incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Continuous Integration: Next Steps
Date Thu, 02 Aug 2012 14:14:14 GMT
Hmm. My experience with Jenkins is nearly zero.  I just created
"crunch-trunk" job, can you edit it?

https://builds.apache.org/job/crunch-trunk/configure



On Thu, Aug 2, 2012 at 9:03 AM, Josh Wills <josh.wills@gmail.com> wrote:

> I get "jwills is missing the Create permission"
>
> On Thu, Aug 2, 2012 at 6:49 AM, Brock Noland <brock@cloudera.com> wrote:
> > According to ldap, we are both in the "hudson-jobadmin" group. There is
> no
> > "New Job" link in the upper left? The link goes here:
> > https://builds.apache.org/view/All/newJob
> >
> > Brock
> >
> > On Thu, Aug 2, 2012 at 12:13 AM, Josh Wills <jwills@cloudera.com> wrote:
> >
> >> Hey Brock,
> >>
> >> I can log in to Jenkins at builds.apache.org, but it doesn't appear
> >> that I can create a new job-- I just have a dashboard that lists Build
> >> History/Project Relationship/etc. Is there some admin bit I'm missing?
> >>
> >> J
> >>
> >> On Tue, Jul 31, 2012 at 7:40 AM, Brock Noland <brock@cloudera.com>
> wrote:
> >> > Done!
> >> >
> >> > Brock
> >> >
> >> > On Tue, Jul 31, 2012 at 9:36 AM, Josh Wills <jwills@cloudera.com>
> wrote:
> >> >> Explicitly emailing Brock, Tom, and Arun, who I believe are all of
> the
> >> PMC
> >> >> chairs on our teams-- guys, could one of your grant me (jwills)
> Jenkins
> >> >> access so I can setup CI?
> >> >>
> >> >> J
> >> >>
> >> >>
> >> >> On Mon, Jul 30, 2012 at 12:59 AM, Rahul <rsharma@xebia.com> wrote:
> >> >>>
> >> >>> I think sonar will be a good way forward.
> >> >>>
> >> >>> As for maven repo for snapshots, there is one snapshots repo from
> >> Apache
> >> >>> at https://repository.apache.org/content/repositories/snapshots/.
> >> >>> I think this is good enough for the purpose and released artifacts
> can
> >> be
> >> >>> uploaded to maven repos.
> >> >>>
> >> >>> regards
> >> >>> Rahul
> >> >>>
> >> >>>
> >> >>> On 29-07-2012 13:32, Matthias Friedrich wrote:
> >> >>>>
> >> >>>> Hi,
> >> >>>>
> >> >>>> over the last three weeks we've been working hard on making
Crunch
> >> >>>> ready for CI. It turned out to be much more work than initially
> >> >>>> expected and I cursed myself more than once for starting this.
In
> the
> >> >>>> end it took four people and many patches to pull this off (thanks
a
> >> >>>> lot for your help, Josh, Rahul, and Gabriel!). In case you're
> >> >>>> interested, these were the issues that lead us to CI readiness:
> >> >>>>
> >> >>>>   * CRUNCH-8: Refactor Crunch's build into a multi-module Maven
> >> project
> >> >>>>   * CRUNCH-17: Split unit tests from integration tests
> >> >>>>   * CRUNCH-18: Improve build stability
> >> >>>>   * CRUNCH-21: Make temporary directory configurable
> >> >>>>   * CRUNCH-24: Make test suite suiteable for continuous integration
> >> >>>>   * CRUNCH-26: Make Scrunch tests suiteable for continuous
> integration
> >> >>>>
> >> >>>> We can now start with basic jobs to detect broken commits by
> running
> >> >>>> the unit and integration test suites. I suggest starting with
an
> >> >>>> on-commit build using "mvn package" and a nightly build using
"mvn
> >> >>>> verify" (which runs the integration test suite). Running the
> >> >>>> integration tests on-commit would be great but may be too
> expensive.
> >> >>>>
> >> >>>> To get started we need access to Apache's Jenkins instance
[1]
> which
> >> >>>> apparently only PMC chairs can grant [2]. I'm new to Apache
> processes,
> >> >>>> but perhaps a mentor can help us out here?
> >> >>>>
> >> >>>>
> >> >>>> Next steps:
> >> >>>>
> >> >>>> CI gives us the ability to regularly run reports to check code
> >> >>>> quality. We already started preparations for this (Maven site
> setup,
> >> >>>> checkstyle config and basic reports), but I think the best
thing
> would
> >> >>>> be to use Apache's Sonar instance [3]. I use Sonar a lot at
work,
> it
> >> >>>> bundles all kinds of code metrics in a nice interface and points
> out
> >> >>>> where refactoring time is best spent. It's the most convenient
way
> of
> >> >>>> tracking code quality over time and catches many issues human
> >> >>>> reviewers usually overlook.
> >> >>>>
> >> >>>> Additional to reports, it's also quite common to deploy nightly
> >> >>>> SNAPSHOT builds to a Maven repository and to publish up-to-date
> Maven
> >> >>>> sites. I'm not sure if this is possible at Apache for legal
reasons
> >> >>>> though.
> >> >>>>
> >> >>>>
> >> >>>> Please let me know where you think we should go next. I know
this
> >> >>>> whole CI episode has been a nuisance, so thanks again for your
> help.
> >> >>>>
> >> >>>> Regards,
> >> >>>>    Matthias
> >> >>>>
> >> >>>> [1] https://builds.apache.org
> >> >>>> [2] http://wiki.apache.org/general/Jenkins
> >> >>>> [3] https://analysis.apache.org
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Director of Data Science
> >> >> Cloudera
> >> >> Twitter: @josh_wills
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Apache MRUnit - Unit testing MapReduce -
> >> http://incubator.apache.org/mrunit/
> >>
> >>
> >>
> >> --
> >> Director of Data Science
> >> Cloudera
> >> Twitter: @josh_wills
> >>
> >
> >
> >
> > --
> > Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message