Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of michael_segel@hotmail.com
 designates 65.55.111.96 as permitted sender)
Message-ID: <BLU0-SMTP1520167217A371D1EF229248F860@phx.gbl>
Content-Type: text/plain; charset="iso-8859-1"
MIME-Version: 1.0 (Mac OS X Mail 6.1 \(1498\))
Subject: Re: HADOOP in Production
From: Michael Segel <michael_segel@hotmail.com>
In-Reply-To: 
 <CACBYqwTnAbHY_Eo5eYd3b-M9NspiQEj3EBDgnhFfeJmW55Ou7A@mail.gmail.com>
Date: Tue, 2 Oct 2012 07:04:20 -0500
Content-Transfer-Encoding: quoted-printable
References: <SNT117-W16EA385B97DC2E63E40257DA870@phx.gbl>
 <CACBYqwTnAbHY_Eo5eYd3b-M9NspiQEj3EBDgnhFfeJmW55Ou7A@mail.gmail.com>
To: user@hadoop.apache.org

Funny that the OP asks about 'real time'...

This comes up quiet often and its always misunderstood.=20

First, when we say 'real time' many take it to mean subjective real =
time.  Real 'real time' would require some sort of RTOS underneath.=20

Second Hadoop is a parallelized framework. You have several components =
that make up Hadoop.  A distributed scheduler, a distributed disk and =
tools to manipulate the data.=20

You can use Hadoop in subjective real time scenarios.=20

One common pattern is to use M/R to process the data, and HBase to =
deliver ad-hoc access to records returning a result in sub second =
response time.=20

I think that there's an upcoming talk at Strata in NY on using Hadoop, =
(HBase and SOLR) to provide real time access.=20

Out side of that, yeah Tom White's book is a great start, however, some =
of the feedback I've heard it that its a dry read.=20
But then again, most technical books are. :-)=20


On Oct 2, 2012, at 6:47 AM, Ruslan Al-Fakikh <metaruslan@gmail.com> =
wrote:

> Hi,
>=20
> There are too many issues to discuss I guess. I would recommend
> reading Hadoop The Definitive Guide by Tom White. There are some
> chapters for the answers.
> Also what did you mean my 'real time"? Hadoop is not designed for
> giving real time results of queries. It is rather for offline data
> analysis, because each query can take minutes or hours to finish.
> AFAIK, HBase provides some real time functionality though.
> For Hadoop automation, you can try Oozie. We are using opswise in our =
company
>=20
> Best Regards
>=20
> On Mon, Oct 1, 2012 at 5:36 PM, yogesh dhari <yogeshdhari@live.com> =
wrote:
>> Hi all,
>>=20
>> I have understood the Hadoop and Hadoop Ecosystem(Pig as ETL, Hive as
>> DataWare house, Sqoop as importing tool). I worked and learned on =
single
>> node cluster with demo data.
>>=20
>> As Hadoop suits best on Unix platform. Please help me to understand =
the
>> requirement form start to finish to use Hadoop in production.
>>=20
>> What would be the things to use Hadoop on real time project.
>>=20
>> like Hadoop automation on Unix, alert of failure process.
>>=20
>> Please put some light on using Hadoop on real time and what =
objectives are
>> recommended.
>>=20
>>=20
>> Thanks & Regards
>> Yogesh Kumar
>>=20
>=20