Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of nitinpawar432@gmail.com
 designates 209.85.128.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CA+kVhsY7uOYrY6EG-U0AETpoLury2miU1R6L0jeEW4g252+ouw@mail.gmail.com>
References: 
 <CA+kVhsY7uOYrY6EG-U0AETpoLury2miU1R6L0jeEW4g252+ouw@mail.gmail.com>
Date: Wed, 6 Feb 2013 03:09:02 +0530
Message-ID: 
 <CAORpBsifJF4vf+hTFxRCyOUnZZthgUzs++sRyU5cp77uL-QGTQ@mail.gmail.com>
Subject: Re: [Hadoop-Help]About Map-Reduce implementation
From: Nitin Pawar <nitinpawar432@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=14dae9cfce1ac5e25b04d50109c4

--14dae9cfce1ac5e25b04d50109c4
Content-Type: text/plain; charset=ISO-8859-1

Hey Mayur,

If you are collecting logs from multiple servers then you can use flume for
the same.

if the contents of the logs are different in format  then you can just use
textfileinput format to read and write into any other format you want for
your processing in later part of your projects

first thing you need to learn is how to setup hadoop
then you can try writing sample hadoop mapreduce jobs to read from text
file and then process them and write the results into another file
then you can integrate flume as your log collection mechanism
once you get hold on the system then you can decide more on which paths you
want to follow based on your requirements for storage, compute time,
compute capacity, compression etc


On Wed, Feb 6, 2013 at 3:01 AM, Mayur Patil <ram.nath241089@gmail.com>wrote:

> Hello,
>
>     I am new to Hadoop. I am doing a project in cloud in which I
>
>     have to use hadoop for Map-reduce. It is such that I am going
>
>     to collect logs from 2-3 machines having different locations.
>
>     The logs are also in different formats such as .rtf .log .txt
>
>     Later, I have to collect and convert them to one format and
>
>     collect to one location.
>
>     So I am asking which module of Hadoop that I need to study
>
>     for this implementation?? Or whole framework should I need
>
>     to study ??
>
>     Seeking for guidance,
>
>     Thank you !!
> --
> *Cheers,*
> *Mayur.*
>


-- 
Nitin Pawar

--14dae9cfce1ac5e25b04d50109c4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hey Mayur,<div><br></div><div style>If you are collecting =
logs from multiple servers then you can use flume for the same.=A0</div><di=
v style><br></div><div style>if the contents of the logs are different in f=
ormat =A0then you can just use textfileinput format to read and write into =
any other format you want for your processing in later part of your project=
s=A0</div>
<div style><br></div><div style>first thing you need to learn is how to set=
up hadoop=A0</div><div style>then you can try writing sample hadoop mapredu=
ce jobs to read from text file and then process them and write the results =
into another file=A0</div>
<div style>then you can integrate flume as your log collection mechanism=A0=
</div><div style>once you get hold on the system then you can decide more o=
n which paths you want to follow based on your requirements for storage, co=
mpute time, compute capacity,=A0compression=A0etc=A0</div>
</div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Wed,=
 Feb 6, 2013 at 3:01 AM, Mayur Patil <span dir=3D"ltr">&lt;<a href=3D"mailt=
o:ram.nath241089@gmail.com" target=3D"_blank">ram.nath241089@gmail.com</a>&=
gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hello,<br><br>=A0 =A0 I am new to Hadoop. I =
am doing a project in cloud in which I=A0<div><br></div><div>=A0 =A0 have t=
o use hadoop for Map-reduce.=A0It is such that I am going=A0</div>
<div><br></div><div>=A0 =A0 to collect logs from 2-3 machines=A0having diff=
erent locations.</div>
<div><br></div><div>=A0 =A0 The logs are also in different formats=A0such a=
s .rtf .log .txt =A0</div><div><br></div><div>=A0 =A0 Later, I have to coll=
ect and convert them=A0to one=A0format and=A0</div><div><br></div><div>=A0 =
=A0 collect=A0to one location.</div>

<div><div>=A0</div><div>=A0 =A0 So I am asking which module of Hadoop that =
I need to study</div><div>=A0 =A0</div><div>=A0 =A0 for this implementation=
?? Or whole framework should I need=A0</div><div><br></div><div>=A0 =A0 to =
study ??</div><div>

<br></div><div>=A0 =A0 Seeking for guidance,</div><div><div><br>=A0 =A0 Tha=
nk you !!</div><span class=3D"HOEnZb"><font color=3D"#888888"><div>--=A0</d=
iv><div><b>Cheers,</b><br><b>Mayur.</b></div>
</font></span></div></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>Nitin Pawar<=
br>
</div>

--14dae9cfce1ac5e25b04d50109c4--