zipkin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem Jiang <willem.ji...@gmail.com>
Subject Re: Infrastructure setup / CI
Date Thu, 06 Sep 2018 08:00:23 GMT
Travis could be more convince the the Jenkins of Apache from my
personal point of view.
If Zipkin already uses Travis, I don't think we need to move to other
CI unless Travis doesn't need our needs.

Just my 2 cents.

Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem

On Thu, Sep 6, 2018 at 3:36 PM Zoltán Nagy <abesto0@gmail.com> wrote:
>
> Thanks for sharing your thoughts, Andrey!
>
> I'm still looking to hear opinions from other Zipkin committers. In the
> meanwhile, my two cents.
>
> Re. moving the GitHub repositories to under the Apache org, this should be
> fine IMHO. But let's wait to hear from Adrian on this. Meta: how are
> questions like this decided under the ASF? Is there a standard decision
> procedure here, or is it up to each project to formalize how decisions like
> this are made? Is there even need for formalizing the process?
>
> At a glance, it looks like ASF Jenkins is running a recent version of
> Jenkins, including pipelines (both declarative and the other one whose name
> I always forget) and multibranch pipelines, with static build nodes
> (documented at
> https://cwiki.apache.org/confluence/display/INFRA/Jenkins+node+labels).
> There's no dynamic (Docker-based or otherwise) provisioning of executors,
> meaning the build environment is what it is, with little room for
> per-project customization. I see 24 nodes running Xenial, with 2 executors
> per node, which is what we'd use, probably exclusively. I didn't find
> analytics on queue times, but looking at the build queue for a couple of
> minutes, it seems to me that queue times are on the minute scale. Is that a
> fair summary?
>
> As for the other options listed on https://ci.apache.org/ (namely Buildbot
> and Gump), I feel we'd be best served by Jenkins of these three; OpenZipkin
> is JVM-heavy, so Jenkins is a “native” CI system. I'm slightly confused
> about why https://ci.apache.org/ doesn't list Travis CI.
>
> My personal stance at this point: infrastructure-wise, the two things we'd
> need to evaluate for feasibility is queue times and environment
> customization (are all our build-time system dependencies available? if
> not, can we replace them with present alternatives, or get them into the
> executors?). I'd probably be doing the majority of the migration work, at
> least in the PoC and initial migration phase, so I want to note that I'm OK
> with that amount of work. At the same time, this would be a big change in
> the way OpenZipkin development is done (Travis to Jenkins), and I'm eager
> to hear from other committers how you all feel about the mental overhead
> (and unavoidable early migration-related breakages) of a migration like
> this. There's value to it, but it's significantly more expensive than
> moving everything to Travis.
>
> Cheers!
>
> On Tue, Sep 4, 2018 at 5:31 PM Andrey Redko <drreta@gmail.com> wrote:
>
> > Hey Zoltán,
> >
> > Indeed, you have brought a lot of great points to discuss!
> >
> > I hope other guys will correct me if I am wrong, but as per
> > https://incubator.apache.org/guides/podling_sourcecontrol.html, the
> > repositories should be hosted by Apache or on Github under Apache
> > organization. The latter should be fine, right?
> >
> > Regarding CI, most project use Jenkins (builds.apache.org), the
> > ci.apache.org is the source for CI infra for ASF. Would it work for you
> > guys (assuming the help with configuring Jenkins will be surely offered).
> >
> > What do you think?
> >
> > Best Regards,
> >     Andriy Redko
> >
> > On Tue, Sep 4, 2018, 4:35 AM Zoltán Nagy <abesto0@gmail.com> wrote:
> >
> > > Hey folks,
> > >
> > > So great to send a mail to a list whose address includes both Zipkin and
> > > Apache :)
> > > Watch out, wall of text incoming. There's a lot of context here.
> > >
> > > After reading the previous mail threads (still before the list), and some
> > > discussion with Andrew (whom Adrian introduced, and was a great help, are
> > > you here, Andrew?), I'm starting to have an outline of infrastructure-y
> > > things that need to happen.
> > >
> > > Most of these are pretty straightforward, and I'll try to capture
> > existing
> > > stated preferences – do shout if I'm telling a lie.
> > >
> > >    - *Git repository hosting*. OpenZipkin people AFAIK would prefer
> > staying
> > >    on GitHub if at all possible. This would imply setting up syncing to
> > ASF
> > >    git hosting. https://gitbox.apache.org/ seems to be a solution to
> > this
> > >    whole can of worms, though I didn't look too deeply yet.
> > >    - Release automation may need to be extended for the required *voting
> > >    process* – or maybe just leave it up to human process, don't change
> > the
> > >    automation.
> > >    - We'll need to set up *automated notification e-mails* about commits,
> > >    releases, that kind of stuff. For commits, it should come out of the
> > box
> > >    with the Git setup, and should be easy to add to the release
> > automation.
> > >    This is probably best done with a separate notifications@ list.
> > >
> > > Now for the hard part: *CI*. Travis seems to be supported by ASF, but
> > > CircleCI is not, and various OpenZipkin projects currently use one, the
> > > other, or even both. ASF apparently also has a hosted Jenkins server
> > (which
> > > I can't find ATM).
> > >
> > > *A question to mentors*: what other CI options are available under the
> > ASF
> > > umbrella? Links to the various solutions or documentation for them would
> > be
> > > especially appreciated, so we can evaluate them properly.
> > >
> > > *A question to OpenZipkin folks*: what's your current take on our CI
> > setup?
> > > Here too, let me share the perspectives I've heard so far, call out any
> > > mistakes I make:
> > >
> > >    - Travis sometimes has stability problems (more often than “rarely”,
> > but
> > >    more rarely than “often”, and more often than Circle)
> > >    - Circle is somewhat more featureful and stable, but is severely
> > limited
> > >    in the number of free executors (and again, is not currently
> > > ASF-approved)
> > >    - The less we need to care about CI, the better – that includes
> > ongoing
> > >    maintenance, mental / process overhead when making changes, and cost
> > of
> > > any
> > >    migration
> > >    - This one's mostly from me: there's a lot of duplication between the
> > >    build definitions of our various projects, not to mention the overhead
> > > of
> > >    “supporting” both Travis and Circle. For example, I know Jenkins has
> > >    standard tools to deduplicate parts of build definitions (namely,
> > shared
> > >    libraries for pipelines); I don't know of similar solutions for Travis
> > > (and
> > >    am happy to be pointed at some / do more research).
> > >
> > > I'd say using a single CI system is more of a “when”, rather than a
> > > “whether” type of question. It then boils down to a trade-off between the
> > > cost of migration, and the perceived value of the new system. Some
> > possible
> > > scenarios:
> > >
> > >    - We decide to minimize the cost of migration, and move Circle builds
> > to
> > >    Travis. Later on, we might look into deduplicating build logic between
> > > jobs.
> > >    - We push for ASF to accept CircleCI (note, with no guarantee of
> > success
> > >    at this point), including funding for more executors if needed (or try
> > > to
> > >    start a collaboration with Circle so that ASF projects get more
> > > executors),
> > >    and migrate Travis jobs to Circle.
> > >    - We decide to maximize perceived value, by choosing a third option
> > >    (non-Travis, non-Circle) that helps DRY our build definitions, while
> > > also
> > >    keeping it easy to make per-project changes as needed. I'd recommend
> > >    leading a project like this with a PoC and evaluating the results
> > there,
> > >    before migrating all N>10 projects and pouring effort into finalizing
> > > the
> > >    shared abstraction layer.
> > >
> > > EOF wall of text. To recap, I'm super interested in what other CI options
> > > are available under ASF (so that we know what to even think about), any
> > > other recommendations our beloved mentors might have given all this, and
> > > preferences / arguments / corrections from Zipkin folks.
> > >
> > > Cheers!
> > >
> > > --
> > > Zoltán Nagy
> > > https://abesto.net
> > >
> >
>
>
> --
> Zoltán Nagy
> https://abesto.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@zipkin.apache.org
For additional commands, e-mail: dev-help@zipkin.apache.org


Mime
View raw message