hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lohit <lohit...@yahoo.com>
Subject Re: parallel hadoop process reading same input file
Date Thu, 28 Aug 2008 16:59:37 GMT
Hi Deepak,
Can you explain what process and what files they are trying to read? If you are talking about
map/reduce tasks reading files on DFS, then, yes parallel reads are allowed. Multiple writers
are not. 

----- Original Message ----
From: Deepak Diwakar <ddeepak4u@gmail.com>
To: core-user@hadoop.apache.org
Sent: Thursday, August 28, 2008 6:06:58 AM
Subject: parallel hadoop process reading same input file


When I am running two hadoop processes in parallel and both process has to
read same file. It fails.
Of course one solution is to keep copy of file into different location so
that accessing simultaneously would not cause any problem. But what if we
don't want to do so because it costs extra space.
Plz do suggest me any suitable solution to this.

Thanks & Regards,

View raw message