Mailing-List: contact user-help@bigtop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@bigtop.apache.org
Received-SPF: pass (nike.apache.org: domain of jayunit100.apache@gmail.com
 designates 209.85.216.46 as permitted sender)
From: Jay Vyas <jayunit100.apache@gmail.com>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)
Subject: Re: BIGTOP-1x branch.. Do we need multitenancy systems?
Message-Id: <2CE9C39B-B544-4E14-9577-477F60A69323@gmail.com>
Date: Wed, 11 Feb 2015 07:56:15 -0500
References: 
 <CACVCA=f6WOiC8sakL66FuTSTn6QV5D1ZpFmXiMesxNnts=J2yQ@mail.gmail.com>
 <20150211011905.GC4732@boudnik.org>
 <B271523E-7B67-4020-95AA-EDF8669159AB@gmail.com>
 <CA+ULb+tQkPyWKFZUT6vQmmUdpkuC6CwkGTLO4y2q=_im6ri1Hg@mail.gmail.com>
 <54DB0F1E.7070902@bmahe.net>
In-Reply-To: <54DB0F1E.7070902@bmahe.net>
To: "user@bigtop.apache.org" <user@bigtop.apache.org>

makes sense to me; This should help me to picture where bigtop is headed for=
 the next several months.

So I guess the answer is "yes : we still beleive in multitenant packaging an=
d systems".

Thanks for all the feedback!

> On Feb 11, 2015, at 3:13 AM, Bruno Mah=C3=A9 <bruno@bmahe.net> wrote:
>=20
>> On 02/10/2015 10:05 PM, Roman Shaposhnik wrote:
>>> On Tue, Feb 10, 2015 at 6:00 PM, RJ Nowling <rnowling@gmail.com> wrote:
>>> Can we articulate the value of packages over tarballs?  In my view, pack=
ages
>>> are useful for managing dependencies and in-place updates.
>> In my view packages are the only way to get into the traditional IT deplo=
yment
>> infrastructures. These are the same infrastructures that don't want to to=
uch
>> Ambari at all, since they are all standardized on Puppet/Chef and traditi=
onal
>> Linux packaging.
>>=20
>> There's quite a few of them out there still, despite all the push of
>> Silicon Valley
>> to get everybody to things like Docker, etc.
>=20
> +1.
>=20
> I like docker and it is a very nice project. But it is not going to be an e=
nd in itself.
> Companies will continue to have various hosts, going from bare metal to di=
fferent clouds providers (SaaS, PaaS...), docker included.
>=20
> Aside from that, using packages provide so many benefits over tarballs:
> * Packages have some metadata so I know what file belong where and how and=
 what version
> * all the dependencies are specified in it. Which makes it easier to reuse=
 even across docker files. This includes system dependencies as well (ex: wh=
o depends on psmisc? why? can it be removed now that we updated Apache Hadoo=
p?)
> * it enables us to respect the Single Responsibility Principe and to satis=
fy everyone, folks using bare metal as well as cloud technologies users
> * some patches may still need to be applied for compatibility/build reason=
s. Using packages makes that easier
> * It provides a deep integration with the system so "it just works". Users=
 are created, initscripts setup, alternatives setup, everything has the righ=
t permissions...
> * It makes it dead easy when I want to build multiple variants of the same=
 image since everything is pulled and setup correctly. If I were to manually=
 unpack tarballs, I would have to take care of that manually and also it wou=
ld take a lot more space than the package equivalent unless I spend a lot of=
 time deleting internal parts of each component. Example: I want hadoop clie=
nt and fuse only for a variant.
>=20
> Note that this could also be done with tarballs as well, but that would re=
quire a lot of duplication of command lines, trials and errors and wouldn't b=
e as maintainable.
>=20
> In conclusion, even if Apache Bigtop was to focus on docker, building pack=
ages would be much better than dropping them and going toward a 'tarball' ap=
proach. Packages would not only be more maintainable, satisfy more use cases=
 but would also provide an abstraction layer so the docker files could focus=
 on the image itself instead of setting up the various combinations of Apach=
e Hadoop components.
> =46rom a 10 000 ft view and in the big lines, docker is not much different=
 than vagrant or boxgrinder. For those tools, having the recipe using the pa=
ckages was simplifying a lot of things and I don't see why it would be diffe=
rent with docker.
>=20
>=20
>>> Related question: what are BigTop's goals? Just integration testing?
>>> Full blown distro targeted at end users? Packaging for others to build d=
istros on top of?
>> All of the above? ;-) Seriously, I think we need to provide a way for con=
sumers
>> of bigdata technology to be able to deploy it in the most efficient
>> way. This means
>> that we are likely to need to embrace different ways of packaging our stu=
ff.
>>=20
>> Thanks,
>> Roman.
> +1 again
>=20
> Another way to put it is to make the Apache Hadoop ecosystem usable.
> That includes making it consumable as well as verifying that it all works t=
ogether.
> Packages have been the main way to consume such artifacts, but we have alw=
ays been opened to other ways (see vagrant and boxgrinder). We even had at s=
ome point a kickstart image to build bootable usb keys with an out of the bo=
x working Apache Hadoop environment :)
>=20
> If tomorrow packages become obsolete, I don't see why we could not drop th=
em. But I think we are still far from that.
>=20
>=20
> Thanks,
> Bruno