incubator-bigtop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <>
Subject Re: Projects that bundle packaging infrastructure
Date Mon, 15 Aug 2011 18:47:21 GMT
I'll provide some of my comments inline, but I really think we should
discuss it more at the upcoming meeting.

On Sat, Aug 13, 2011 at 3:39 PM, Bruno Mahé <> wrote:
> I think it is too early to think about that and we should wait for these
> packaging efforts to be more battle tested. Until then, we should focus
> on what is already in BigTop.

Agreed. It is definitely NOT something we need to address right away
if for no other reason but simply because I don't think any of the
projects I was referring to has had a formal release with packaging
infrastructure. It is all in trunk at this point.

However, I would Bigtop to not just passively react to what happens
when they release, but rather work with those communities ahead
of time.

That's why I was really hoping for Owen/Alan/Eric Y. to chime in on this
discussion since they seem to be intimately involved in providing
packaging infrastructure for those projects.

> From a technical point of view, last time I checked, Hadoop and Pig
> packaging efforts only concern RPMs for RHEL/CentOS-like platform (I
> remember seeing a few things that wouldn't work on something else than
> RHEL/CentOS-like, maybe it has changed since).  Whereas BigTop comes
> from CDH and even though it comes with some history and cruft, it works
> and has been deployed in a wide variety of production clusters across
> broad environment and GNU/Linux distributions. So I would rather work on
> cleaning up, improving or bringing new features to BigTop instead of
> helping other packaging efforts catching up on what BigTop has been
> doing since day one.

I tend to agree with that assessment. That said, given the different
choices in packaging that were implement in Hadoop/Pig I think
we should really find out WHY they were implemented that way.
Perhaps there was some kind of customer feedback that we
really have to listen to. So again, it is rather crucial for us to get
Owen/Alan/Eric Y. into this discussion.

I'm ambivalent on whether packaging infrastructure should be kept
upstream or reside in Bigtop proper. After all, quite a few OS projects
still have top-level debian/ subdirectory in their tarballs.

That said, I'd really, really hate to see us diverge on packaging
without any good reason behind it. In my mind, the beauty of
Bigtop is exactly its goal of being a place where packaging
and integration happens for the benefit of all sorts of different
customers ranging from individuals all the way to Linux packagers.
It simply would be a disservice to our customer base to have
Apache releases of Hadoop packaged differently from, lets
say Ubuntu.

> (*) I am also lobbying for building and testing our packaging against
> the trunk of each project so we can pro-actively ensure releases have a
> good enough quality for us, rather than waiting for a release to go out,
> file bug/patches and wait for the next release to be able to use it (if
> no other issues have been introduced). But all of this is pointless
> until we get some build going on.



View raw message