crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: Continuous Integration: Next Steps
Date Thu, 02 Aug 2012 05:13:27 GMT
Hey Brock,

I can log in to Jenkins at builds.apache.org, but it doesn't appear
that I can create a new job-- I just have a dashboard that lists Build
History/Project Relationship/etc. Is there some admin bit I'm missing?

J

On Tue, Jul 31, 2012 at 7:40 AM, Brock Noland <brock@cloudera.com> wrote:
> Done!
>
> Brock
>
> On Tue, Jul 31, 2012 at 9:36 AM, Josh Wills <jwills@cloudera.com> wrote:
>> Explicitly emailing Brock, Tom, and Arun, who I believe are all of the PMC
>> chairs on our teams-- guys, could one of your grant me (jwills) Jenkins
>> access so I can setup CI?
>>
>> J
>>
>>
>> On Mon, Jul 30, 2012 at 12:59 AM, Rahul <rsharma@xebia.com> wrote:
>>>
>>> I think sonar will be a good way forward.
>>>
>>> As for maven repo for snapshots, there is one snapshots repo from Apache
>>> at https://repository.apache.org/content/repositories/snapshots/.
>>> I think this is good enough for the purpose and released artifacts can be
>>> uploaded to maven repos.
>>>
>>> regards
>>> Rahul
>>>
>>>
>>> On 29-07-2012 13:32, Matthias Friedrich wrote:
>>>>
>>>> Hi,
>>>>
>>>> over the last three weeks we've been working hard on making Crunch
>>>> ready for CI. It turned out to be much more work than initially
>>>> expected and I cursed myself more than once for starting this. In the
>>>> end it took four people and many patches to pull this off (thanks a
>>>> lot for your help, Josh, Rahul, and Gabriel!). In case you're
>>>> interested, these were the issues that lead us to CI readiness:
>>>>
>>>>   * CRUNCH-8: Refactor Crunch's build into a multi-module Maven project
>>>>   * CRUNCH-17: Split unit tests from integration tests
>>>>   * CRUNCH-18: Improve build stability
>>>>   * CRUNCH-21: Make temporary directory configurable
>>>>   * CRUNCH-24: Make test suite suiteable for continuous integration
>>>>   * CRUNCH-26: Make Scrunch tests suiteable for continuous integration
>>>>
>>>> We can now start with basic jobs to detect broken commits by running
>>>> the unit and integration test suites. I suggest starting with an
>>>> on-commit build using "mvn package" and a nightly build using "mvn
>>>> verify" (which runs the integration test suite). Running the
>>>> integration tests on-commit would be great but may be too expensive.
>>>>
>>>> To get started we need access to Apache's Jenkins instance [1] which
>>>> apparently only PMC chairs can grant [2]. I'm new to Apache processes,
>>>> but perhaps a mentor can help us out here?
>>>>
>>>>
>>>> Next steps:
>>>>
>>>> CI gives us the ability to regularly run reports to check code
>>>> quality. We already started preparations for this (Maven site setup,
>>>> checkstyle config and basic reports), but I think the best thing would
>>>> be to use Apache's Sonar instance [3]. I use Sonar a lot at work, it
>>>> bundles all kinds of code metrics in a nice interface and points out
>>>> where refactoring time is best spent. It's the most convenient way of
>>>> tracking code quality over time and catches many issues human
>>>> reviewers usually overlook.
>>>>
>>>> Additional to reports, it's also quite common to deploy nightly
>>>> SNAPSHOT builds to a Maven repository and to publish up-to-date Maven
>>>> sites. I'm not sure if this is possible at Apache for legal reasons
>>>> though.
>>>>
>>>>
>>>> Please let me know where you think we should go next. I know this
>>>> whole CI episode has been a nuisance, so thanks again for your help.
>>>>
>>>> Regards,
>>>>    Matthias
>>>>
>>>> [1] https://builds.apache.org
>>>> [2] http://wiki.apache.org/general/Jenkins
>>>> [3] https://analysis.apache.org
>>>
>>>
>>
>>
>>
>> --
>> Director of Data Science
>> Cloudera
>> Twitter: @josh_wills
>>
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



-- 
Director of Data Science
Cloudera
Twitter: @josh_wills

Mime
View raw message