Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 861EF1869E for ; Tue, 2 Jun 2015 02:27:49 +0000 (UTC) Received: (qmail 85563 invoked by uid 500); 2 Jun 2015 02:27:44 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 85438 invoked by uid 500); 2 Jun 2015 02:27:44 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 85428 invoked by uid 99); 2 Jun 2015 02:27:44 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Jun 2015 02:27:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C3780CA954 for ; Tue, 2 Jun 2015 02:27:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.792 X-Spam-Level: * X-Spam-Status: No, score=1.792 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H2=-1.108, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id NkYqeH6f_Rc9 for ; Tue, 2 Jun 2015 02:27:38 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id C97EC24CEA for ; Tue, 2 Jun 2015 02:27:37 +0000 (UTC) Received: by ieclw1 with SMTP id lw1so28159658iec.3 for ; Mon, 01 Jun 2015 19:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=7/kyYm6lIYs9cQtH/JWhvCfwFTwmTSgpwTIC/b9w2MU=; b=cSzAJxYywixmYCFbM5VbORPnagZXXKemAJNFezUTWgBa8HXnN0sOiv11tQRxqRVEVr 9km5L11NIr4d6UPL1InV+VhMuEeYpE7/9sBjxxBt5JhU3ZjHwK+IF3g5jeOMHffqIA0z EKc8L0ukNBi1BHAxMrnVTaSishpg3jCWK4gKzQgxdYrARXkbQF+bDFaZp4x4m0gEpQQt J5qRXDzJgK8nTF91FlEwb/6yKXhrs2Tw1WbvHTDRGNRqex5JqtcRsAf4cKxlBnXxvZzA P1FZFZARqiUD5wp985JafoGiInQ9dVimWxF5plq7v9WsBjkuMkcwwuBbJQMQCnClxS0J Ecbw== MIME-Version: 1.0 X-Received: by 10.107.170.80 with SMTP id t77mr24128234ioe.31.1433212056261; Mon, 01 Jun 2015 19:27:36 -0700 (PDT) Received: by 10.64.167.162 with HTTP; Mon, 1 Jun 2015 19:27:36 -0700 (PDT) In-Reply-To: References: Date: Mon, 1 Jun 2015 19:27:36 -0700 Message-ID: Subject: Re: a non-commerial distribution of hadoop ecosystem? From: Demai Ni To: "user@hadoop.apache.org" Cc: "dev@bigtop.apache.org" Content-Type: multipart/alternative; boundary=001a11415d58805e1a05177faf34 --001a11415d58805e1a05177faf34 Content-Type: text/plain; charset=UTF-8 Andrew, great to hear that you are also using BigTop. I will surely try it out, to replace my (a little bit) old CDH cluster. :-) cheers Demai On Mon, Jun 1, 2015 at 5:29 PM, Andrew Purtell wrote: > Bigtop, in a nutshell, is a non-commercial multi-stakeholder Apache > project that produces a build framework that takes as input source from > Hadoop and related big data projects and produces as output OS native > packages for installation and management - certainly, a distribution of the > Hadoop ecosystem - coupled with a suite of integration tests for ensuring > the distribution components are working well together, coupled with a suite > of Puppet scripts for post-deploy configuration management. It's a rather > large nutshell. (Smile) Bigtop distribution packages are supported by > Cask's Coopr (coopr.io) and I think to some extent by Ambari (haven't > tried it). > > I've personally used Bigtop for years to produce several custom Hadoop > distributions. For this purpose it is a great tool. > > Please mail user@bigtop.apache.org if you would like to know more, we'd > love to talk with you. > > > On Jun 2, 2015, at 7:16 AM, Demai Ni wrote: > > Chris and Roman, > > many thanks for the quick response. I will take a look at bigtop. > Actually, I heard about it, but thought it is a installation framework, > instead of a hadoop distribution. Now I am looking at the BigTop 0.7.0 > hadoop instruction, which probably will work fine for my needs. Appreciate > the pointer. > > Roman, I will ping you off list for ODP. I was hoping ODP will be the one > for me. Well, in reality, it is owned by a few companies, at least not by > ONE company. :-) It is fine with me, as long as ODP is open to be used by > others. I am just having trouble to find document/installation info of the > ODP. maybe I should google harder? :-) > > Demai > > > On Mon, Jun 1, 2015 at 1:46 PM, Roman Shaposhnik wrote: > >> On Mon, Jun 1, 2015 at 1:37 PM, Demai Ni wrote: >> > My question is besides the commercial distributions: CDH(Cloudera) , >> HDP >> > (Horton work), and others like Mapr, IBM... Is there a distribution >> that is >> > NOT owned by a company? I am looking for something simple for cluster >> > configuration/installation for multiple components: hdfs, yarn, >> zookeeper, >> > hive, hbase, maybe Spark. Surely, for a well-experience person(not me), >> > he/she can build the distribution from Apache releases. Well, I am more >> > interested on building application on top of it, and hopefully to find >> one >> > packed them together. >> >> Apache Bigtop (CCed) aims at delivering a 100% open and >> community-driven distribution of big data management technologies >> around Apache Hadoop. Same as, for example, what Debian is trying >> to do for Linux. >> >> > BTW, I don't need the latest releases like other commercial distribution >> > offered. I am also looking into the ODP(the open data platform), but >> that >> > project is kind of quiet after the initial Feb announcement. >> >> Feel free to ping me off list if you want more details on ODP. >> >> Thanks, >> Roman. >> > > --001a11415d58805e1a05177faf34 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Andrew,

great to hear that you are also= using BigTop. I will surely try it out, to replace my (a little bit) old C= DH cluster. :-)=C2=A0

cheers

<= div>Demai=C2=A0

On Mon, Jun 1, 2015 at 5:29 PM, Andrew Purtell = <andrew.pu= rtell@gmail.com> wrote:
Bigtop, in a nutshell, is a non-commercial multi-stakeh= older Apache project that produces a build framework that takes as input so= urce from Hadoop and related big data projects and produces as output OS na= tive packages for installation and management - certainly, a distribution o= f the Hadoop ecosystem - coupled with a suite of integration tests for ensu= ring the distribution components are working well together, coupled with a = suite of Puppet scripts for post-deploy configuration management. It's = a rather large nutshell. (Smile) =C2=A0Bigtop distribution packages are sup= ported by Cask's Coopr (c= oopr.io) and I think to some extent by Ambari (haven't tried it).

I've personally used Bigtop for years to produc= e several custom Hadoop distributions. For this purpose it is a great tool.= =C2=A0

Please mail user@bigtop.apache.org if you would like to know more, we&#= 39;d love to talk with you.


On= Jun 2, 2015, at 7:16 AM, Demai Ni <nidmgg@gmail.com> wrote:

Chris and Roman,

man= y thanks for the quick response.=C2=A0 I will take a look at bigtop. Actual= ly, I heard about it, but thought it is a installation framework, instead o= f a hadoop distribution. Now I am looking at the BigTop 0.7.0 hadoop instru= ction, which probably will work fine for my needs. Appreciate the pointer.<= /div>

Roman, I will ping you off list for ODP. I was hop= ing ODP will be the one for me. Well, in reality, it is owned by a few comp= anies, at least not by ONE company. :-) =C2=A0It is fine with me, as long a= s ODP is open to be used by others. I am just having trouble to find docume= nt/installation info of the ODP. maybe I should google harder? :-)

Demai=C2=A0


On Mon, Jun 1, 2015 at 1:46 PM, Roman= Shaposhnik <rvs@apache.org> wrote:
On Mon, Jun 1, 2015 at 1:37 PM, Demai Ni <nidmgg@gmail.com> wrote:
> My question is besides the commercial distributions: CDH(Cloudera)=C2= =A0 , HDP
> (Horton work), and others like Mapr, IBM... Is there a distribution th= at is
> NOT owned by a company?=C2=A0 I am looking for something simple for cl= uster
> configuration/installation for multiple components: hdfs, yarn, zookee= per,
> hive, hbase, maybe Spark. Surely, for a well-experience person(not me)= ,
> he/she can build the distribution from Apache releases. Well, I am mor= e
> interested on building application on top of it, and hopefully to find= one
> packed them together.

Apache Bigtop (CCed) aims at delivering a 100% open and
community-driven distribution of big data management technologies
around Apache Hadoop. Same as, for example, what Debian is trying
to do for Linux.

> BTW, I don't need the latest releases like other commercial distri= bution
> offered.=C2=A0 I am also looking into the ODP(the open data platform),= but that
> project is kind of quiet after the initial Feb announcement.

Feel free to ping me off list if you want more details on ODP.

Thanks,
Roman.


--001a11415d58805e1a05177faf34--