Return-Path: X-Original-To: apmail-bigtop-user-archive@www.apache.org Delivered-To: apmail-bigtop-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DDE3017F26 for ; Tue, 3 Mar 2015 17:24:15 +0000 (UTC) Received: (qmail 79618 invoked by uid 500); 3 Mar 2015 17:24:04 -0000 Delivered-To: apmail-bigtop-user-archive@bigtop.apache.org Received: (qmail 79550 invoked by uid 500); 3 Mar 2015 17:24:04 -0000 Mailing-List: contact user-help@bigtop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@bigtop.apache.org Delivered-To: mailing list user@bigtop.apache.org Received: (qmail 79540 invoked by uid 99); 3 Mar 2015 17:24:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Mar 2015 17:24:04 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jayunit100.apache@gmail.com designates 74.125.82.48 as permitted sender) Received: from [74.125.82.48] (HELO mail-wg0-f48.google.com) (74.125.82.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Mar 2015 17:23:58 +0000 Received: by wggy19 with SMTP id y19so41350617wgg.13 for ; Tue, 03 Mar 2015 09:22:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=E+GcW7DFNOHaW8KBGqbdL0i1o4IhjpYRIVPdoxNAZB0=; b=aJkb3pDmrrTUVr42Gad5UKieQgguZYZcyiO5ldj6AdZlmPyFC22qjhmuTg0n1fih48 2+7f2ymYUp7WygNBG4yG09J51Uv5jF/xrRWDhALlcz6RUp4yNUbMBjTI6mgRWGc4in4C 3bXV9k5ImeSaxAkwMsIxFkl2i6Qi5rHhDOpyGLwtDMCFYECzWM43bHwnpCnKxh0OmWQK 2s1+dqdCy4+JFoT9TGfN48u7g+P8UmzE2LSVn6R8UHXcg23Zh/vaTy1XQHhjgmUTFVwn EPe3GE3iVjzcPuYjv9S1dojR+4FTfhndLe+8rgHXUanRsmSoihLX4eWFZhOHJbLiRhh8 H3sA== MIME-Version: 1.0 X-Received: by 10.180.78.202 with SMTP id d10mr47798523wix.25.1425403372795; Tue, 03 Mar 2015 09:22:52 -0800 (PST) Received: by 10.27.175.216 with HTTP; Tue, 3 Mar 2015 09:22:52 -0800 (PST) In-Reply-To: <20150303171648.GZ1657@boudnik.org> References: <20150303171648.GZ1657@boudnik.org> Date: Tue, 3 Mar 2015 12:22:52 -0500 Message-ID: Subject: Re: Update on meetup? From: jay vyas To: "user@bigtop.apache.org" Content-Type: multipart/alternative; boundary=f46d043c0848b27ea8051065955a X-Virus-Checked: Checked by ClamAV on apache.org --f46d043c0848b27ea8051065955a Content-Type: text/plain; charset=UTF-8 absolutely supportive of olaf's ideas to make the docker support more platform agnostic. We don't want to tie folks down - bigtop's mission is to be open to solutions from all vendors ! On Tue, Mar 3, 2015 at 12:16 PM, Konstantin Boudnik wrote: > Olaf, > > that's really pity that our docker stuff is CentOS specific, which is I > guess > expected when RedHat guys are working on them ;) Do you think they can be > made > more generic? After all, we use the same Puppet for any supported > platform, so > I don't see why we shouldn't be able to have docker env. that works for any > containerised OS. > > Cos > > On Tue, Mar 03, 2015 at 09:57AM, Olaf Flebbe wrote: > > Hi > > > > Too bad that I missed meetup and start of discussion, because I only > monitored dev list... > > > > We deploy our testcluster on a single machine as pure lxc containers. > Using > > a hiera supported puppet michel weiser is trying to merge with bigtop. > > > > Imho one solution does not fits all. > > > > In productiom: On a cluster of machines you would deploy bigtop on bare > > metal, not sacrifing runtime performance for anything. > > > > In a longer running testcluster on a maschine: > > We run a traditional os in container (i.e. We run systemd formely known > as > > init in it). This has the big advantage that we can test long running > things > > like kerberos tickets expiring happening on a real bare metal cluster. > Right > > now we have journalling, with tez on hive including hue and are happy > right > > now when we can switch on kerberos authentication. > > > > For smoke tests the vagrant/docker scripts are really cool, i looked at > it > > yesterday the first time. Too bad they are deeply bound to centos, I had > to > > change many thing to get it run on debian. > > > > > > Docker is more centered > > > > I looked into the vm puppet scripts and > > > > > > Sent with SMTP > > > > Am 30 Jan 2015 um 20:10 schrieb jay vyas >: > > > > > Roman youre right, a real docker container doesn't do things the way > we currently do in bigtop. > > > > > > If we embrace docker, we will be able to really drop alot of weight. > > > > > > Maybe the experimental bigtop 1x can deploy pure tarballs from apache > distributions on apache mesos for the orchestration ? > > > > > > That would be verrrrrrrryyyyyyyyyyyy interesting ..... > > > > > > On Fri, Jan 30, 2015 at 12:45 PM, Roman Shaposhnik < > roman@shaposhnik.org > wrote: > > > On Thu, Jan 29, 2015 at 7:09 AM, Evans Ye evansye@apache.org>> wrote: > > > > I really like the idea to deploy hadoop stack in docker containers. > > > > Just want to catch up with you guys, are you planning to ship docker > images > > > > as a drop and run cluster deployment? or no? Anyhow that was just my > guess, > > > > I'm interesting in the challenge of this cool stuff :) > > > > > > That's a $64k question. On one hand, a kosher docker container has just > > > one single service inside of it. We can do that very easily. Then the > real > > > questions kick in: how do you orchestrate those, how do you provide > > > them with configuration, etc. Seems like you still need some kind of > > > a system like Amabari, etc. > > > > > > Thanks, > > > Roman. > > > > > > > > > > > > -- > > > jay vyas > > -- jay vyas --f46d043c0848b27ea8051065955a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
absolutely supportive of olaf's ideas to make the dock= er support more platform agnostic.=C2=A0 We don't want to tie folks dow= n - bigtop's mission is to be open to solutions from all vendors !
<= div>
On Tue, Mar 3= , 2015 at 12:16 PM, Konstantin Boudnik <cos@apache.org> wrote:<= br>
Olaf,

that's really pity that our docker stuff is CentOS specific, which is I= guess
expected when RedHat guys are working on them ;) Do you think they can be m= ade
more generic? After all, we use the same Puppet for any supported platform,= so
I don't see why we shouldn't be able to have docker env. that works= for any
containerised OS.

Cos

On Tue, Mar 03, 2015 at 09:57AM, Olaf Flebbe wrote:
> Hi
>
> Too bad that I missed meetup and start of discussion, because I only m= onitored dev list...
>
> We deploy our testcluster on a single machine as pure lxc containers. = Using
> a hiera supported puppet michel weiser is trying to merge with bigtop.=
>
> Imho one solution does not fits all.
>
> In productiom: On a cluster of machines you would deploy bigtop on bar= e
> metal, not sacrifing runtime performance for anything.
>
> In a longer running testcluster on a maschine:
> We run a traditional os in container (i.e. We run systemd formely know= n as
> init in it). This has the big advantage that we can test long running = things
> like kerberos tickets expiring happening on a real bare metal cluster.= Right
> now we have journalling, with tez on hive including hue and are happy = right
> now when we can switch on kerberos authentication.
>
> For smoke tests the vagrant/docker scripts are really cool, i looked a= t it
> yesterday the first time. Too bad they are deeply bound to centos, I h= ad to
> change many thing to get it run on debian.
>
>
> Docker is more centered
>
> I looked into the vm puppet scripts and
>
>
> Sent with SMTP
>
> Am 30 Jan 2015 um 20:10 schrieb jay vyas <jayunit100.apache@gmail.com <mailto:jayunit100.apache@gmail.com= >>:
>
> > Roman youre right, a real docker container doesn't do things = the way we currently do in bigtop.
> >
> > If we embrace docker, we will be able to really drop alot of weig= ht.
> >
> > Maybe the experimental=C2=A0 bigtop 1x can deploy pure tarballs f= rom apache distributions on apache mesos for the orchestration ?
> >
> > That would be verrrrrrrryyyyyyyyyyyy interesting .....
> >
> > On Fri, Jan 30, 20= 15 at 12:45 PM, Roman Shaposhnik <roman@shaposhnik.org <mailto:roman@shaposhnik.org>> wrote:
> > On Thu, Jan 29, 2015 at 7:09 AM, Evans Ye <evansye@apache.org <mailto:evansye@apache.org>> wrote:
> > > I really like the idea to deploy hadoop stack in docker cont= ainers.
> > > Just want to catch up with you guys, are you planning to shi= p docker images
> > > as a drop and run cluster deployment? or no? Anyhow that was= just my guess,
> > > I'm interesting in the challenge of this cool stuff :) > >
> > That's a $64k question. On one hand, a kosher docker containe= r has just
> > one single service inside of it. We can do that very easily. Then= the real
> > questions kick in: how do you orchestrate those, how do you provi= de
> > them with configuration, etc. Seems like you still need some kind= of
> > a system like Amabari, etc.
> >
> > Thanks,
> > Roman.
> >
> >
> >
> > --
> > jay vyas




--
jay vyas
--f46d043c0848b27ea8051065955a--