hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rab ra <rab...@gmail.com>
Subject Muliple map writing into same hdfs file
Date Thu, 10 Jul 2014 05:42:54 GMT
hello



I have one use-case that spans multiple map tasks in hadoop environment. I
use hadoop 1.2.1 and with 6 task nodes. Each map task writes their output
into a file stored in hdfs. This file is shared across all the map tasks.
Though, they all computes thier output but some of them are missing in the
output file.



The output file is an excel file with 8 parameters(headings). Each map task
is supposed to compute all these 8 values, and save it as soon as it is
computed. This means, the programming logic of a map task opens the file,
writes the value and close, 8 times.



Can someone give me a hint on whats going wrong here?



Is it possible to make more than one map task to write in a shared file in
HDFS?

Mime
View raw message