hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lohit <lohit...@yahoo.com>
Subject Re: Streaming a big file using hadoop
Date Thu, 14 Feb 2008 18:41:52 GMT
What kind of streaming are you trying. If you are running a map/reduce job and providing this
as input then hadoop by default splits this file and streams to map/reduce script. Even if
the script is not splittable hadoop can stream the file but I am assuming your file is made
up of records. For example a 4G text file would be line separated. 
Thanks,
Lohit

----- Original Message ----
From: "xavier.quintuna@orange-ftgroup.com" <xavier.quintuna@orange-ftgroup.com>
To: core-dev@hadoop.apache.org
Sent: Thursday, February 14, 2008 10:26:16 AM
Subject: Streaming a big file using hadoop


Hi 
there

Is 
it 
possible 
to 
stream 
a 
big 
file 
(4G) 
using 
the 
hdfs. 
I 
really
appreciate 
any 
help

Xavier




Mime
View raw message