Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B7010100ED for ; Mon, 28 Dec 2015 03:12:53 +0000 (UTC) Received: (qmail 9221 invoked by uid 500); 28 Dec 2015 03:12:49 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 9109 invoked by uid 500); 28 Dec 2015 03:12:49 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 9097 invoked by uid 99); 28 Dec 2015 03:12:49 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Dec 2015 03:12:49 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 9FCE61A1265 for ; Mon, 28 Dec 2015 03:12:48 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id ooi3WLBYm_gf for ; Mon, 28 Dec 2015 03:12:47 +0000 (UTC) Received: from mail-ig0-f177.google.com (mail-ig0-f177.google.com [209.85.213.177]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id A347D215DD for ; Mon, 28 Dec 2015 03:12:46 +0000 (UTC) Received: by mail-ig0-f177.google.com with SMTP id to4so137776310igc.0 for ; Sun, 27 Dec 2015 19:12:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=iDqGi+Az5xuJi2gzbGdwr4L+ss1BB7HgUf1oyBU5GX4=; b=XYKyunhHNfUywViZxfZ/yGU75XsI1uvfe4M3bVs9hnU8iD01IGQ8ff7EM7j76/Fxoe qNejkolsAM3jSzrkkpq9HA0q4sOm7tNW/cvnxPRu/iHLSiL9pMOkczF1gXYLqbF9W3RL r7r1rB9uVR6a215GDv/tlDu9+7etDGJVsEI/LE3c0LAdS/QiZb6npUkEu3TXHOVm7P59 wIAcocQS+EtM5bGJ+Kr0VfufUdPHpTYni32qV+QE4rVQhMBx+WeDT10a5g+NkKn+J/rz iDt7a9xUoVeSrerfU5YIaSIql1RiS0YQNy5CkkEF/eomISfJWGxYTY6XfYMvyuDnPy5P wYpg== X-Received: by 10.50.183.38 with SMTP id ej6mr10088333igc.56.1451272359152; Sun, 27 Dec 2015 19:12:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.50.181.199 with HTTP; Sun, 27 Dec 2015 19:12:09 -0800 (PST) In-Reply-To: References: From: daemeon reiydelle Date: Sun, 27 Dec 2015 19:12:09 -0800 Message-ID: Subject: Re: Hadoop reference architecture To: Steve Baker Cc: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a1136b1ee7095370527ecad5a --001a1136b1ee7095370527ecad5a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable "Best" practices are either so generic as to be mostly useless, or dependent on very specific business processes, SLA's, and OLA's relevant to different data models, etc. If you can identify specific business use cases? Even basic attributes change the model: Containers on CoreOS? White boxes? SSD? Mixed HDD/SSD? AWS? OpenStack? VMWare? High volume ingest with long retentions? Fine grain, well structured data for which data mining is desired or log based analytics? *.......* *=E2=80=9CLife should not be a journey to the grave with the intention of a= rriving safely in apretty and well preserved body, but rather to skid in broadside in a cloud of smoke,thoroughly used up, totally worn out, and loudly proclaiming =E2=80=9CWow! What a Ride!=E2=80=9D - Hunter ThompsonDaemeon C.= M. ReiydelleUSA (+1) 415.501.0198London (+44) (0) 20 8144 9872* On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker wrote: > Dear All > > Apology if this is not the right mailing list to ask this question or if > the request is too simple. > > We are in process of designing a Hadoop cluster and we wonder if you coul= d > share or direct us to a reference architecture design document or somethi= ng > like that (Physical design). We have looked through Apache Hadoop docume= nt > but we were unable to find the required info. However, there are some > vendors specific documents on the web but we are after general document o= r > best practice to design a Hadoop cluster. > > Many Thanks > > Steve > --001a1136b1ee7095370527ecad5a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
"Best" practices are either = so generic as to be mostly useless, or dependent on very specific business = processes, SLA's, and OLA's relevant to different data models, etc.=

If you can identify specific business use= cases?

Even basic attributes change the m= odel: Containers on CoreOS? White boxes? SSD? Mixed HDD/SSD? AWS? OpenStack= ? VMWare?

High volume ingest with long ret= entions? Fine grain, well structured data for which data mining is desired = or log based analytics?


.......
= =E2=80=9CLife = should not be a journey to the grave with the intention of arriving safely in a
pretty and well preserved body, but rather to skid in broadside in a cloud of smoke,
thoroughly used up, totally worn out, and loudly proclaiming =E2=80=9CWow! What a Ride!=E2=80=9D

- = Hunter Thompson

Daemeon C.M. Reiydelle
USA (+1) 415.501.0198
L= ondon (+44) (0) 20 8144 9872

On Sun, Dec 27, 2015 at 7:39 AM, Steve Baker= <baker.steve171@gmail.com> wrote:
Dear All

A= pology if this is not the right=C2=A0mailing list=C2=A0to ask this question= or if the request is too simple.

We are in proces= s of designing a Hadoop cluster and we wonder if you could share or direct = us to a reference architecture design document or something like that=C2=A0= (Physical design). We have looked through Apache Hadoop document but we we= re unable to find the required info. However, there are some vendors specif= ic documents on the web but we are after general document or best practice = to design a Hadoop cluster.

Many Thanks

Steve
<= /font>

--001a1136b1ee7095370527ecad5a--