Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of dechouxb@gmail.com designates
 209.85.215.46 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <BAY177-W379BBB818D13469704A4CB14E0@phx.gbl>
References: 
 <CAJpToqdiK4cd-51RkoD8ieMbJmcuTsZjiDy+yhPML6R-W0CK+A@mail.gmail.com>
	<BAY177-W179353C542747B10E710D9B1430@phx.gbl>
	<BAY177-W352001DEC51F30E5F50BEBB1430@phx.gbl>
	<11564200.Tc0LUDiuXJ@f6-dat6-306-27>
	<BAY177-W379BBB818D13469704A4CB14E0@phx.gbl>
Date: Fri, 9 May 2014 15:46:24 +0200
Message-ID: 
 <CAO6W-2dWX4zGkV9pSUySfE_TdtpDomTAWs7U9QcCy+O_kbEjhw@mail.gmail.com>
Subject: Re: Realtime sensor's tcpip data to hadoop
From: Bertrand Dechoux <dechouxb@gmail.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>, user@flume.apache.org
Content-Type: multipart/alternative; boundary=001a113457dcd1123e04f8f7d263

--001a113457dcd1123e04f8f7d263
Content-Type: text/plain; charset=UTF-8

Flume is indeed something you should look into. 'Log files' is a
simplification. Flume really handles events and yes logs are a common kind
of event but not the only one.

Regards

Bertrand

On Wed, May 7, 2014 at 4:48 AM, Alex Lee <eliyart@hotmail.com> wrote:

> Sensors' may send tcpip data to server. Each sensor may send tcpip data
> like a stream to the server, the quatity of the sensors and the data rate
> of the data is high.
>
> Firstly, how the data from tcpip can be put into hadoop. It need to do
> some process and store in hbase. Does it need through save to data files
> and put into hadoop or can be done in some direct ways from tcpip. Is there
> any software module can take care of this. Searched that Ganglia Nagios and
> Flume may do it. But when looking into details, ganglia and nagios are
> more for monitoring hadoop cluster itself. Flume is for log files.
>
> Secondly, if the total network traffic from sensors are over the limit of
> one lan port, how to share the loads, is there any component in hadoop to
> make this done automatically.
>
> Any suggestions, thanks.
>

--001a113457dcd1123e04f8f7d263
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Flume is indeed something you should look into. &#39;Log f=
iles&#39; is a simplification. Flume really handles events and yes logs are=
 a common kind of event but not the only one.<div><br></div><div>Regards</d=
iv>
<div><br></div><div>Bertrand</div><div class=3D"gmail_extra"><br><div class=
=3D"gmail_quote">On Wed, May 7, 2014 at 4:48 AM, Alex Lee <span dir=3D"ltr"=
>&lt;<a href=3D"mailto:eliyart@hotmail.com" target=3D"_blank">eliyart@hotma=
il.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">


<div><div dir=3D"ltr">Sensors&#39;=C2=A0may send tcpip data to server. Each=
 sensor may send tcpip=C2=A0data like a stream to the server, the quatity o=
f the sensors and the data rate of the data is high.<br>=C2=A0<br>Firstly, =
how the=C2=A0data from tcpip can be put into hadoop. It need to do some pro=
cess and store in hbase. Does it need through save to=C2=A0data files and p=
ut into hadoop=C2=A0or can be done in some=C2=A0direct ways from tcpip. Is =
there any software module can take care of this.=C2=A0Searched that=C2=A0Ga=
nglia=C2=A0Nagios and Flume may do it. But when looking into details, gangl=
ia and nagios are more=C2=A0for monitoring hadoop cluster itself. Flume is =
for log files.<br>
=C2=A0<br>Secondly, if=C2=A0the=C2=A0total network traffic from sensors are=
 over the limit of one lan port, how to share the loads, is there any compo=
nent in hadoop to make this done automatically.<br>=C2=A0<br>Any suggestion=
s, thanks.<br> 		 	   		  </div>
</div>
</blockquote></div><br></div></div>

--001a113457dcd1123e04f8f7d263--