hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ondřej Klimpera <klimp...@fit.cvut.cz>
Subject Creating and working with temporary file in a map() function
Date Sat, 07 Apr 2012 20:44:51 GMT

I would like to ask you if it is possible to create and work with a 
temporary file while in a map function.

I suppose that map function is running on a single node in Hadoop 
cluster. So what is a safe way to create a temporary file and read from 
it in one map() run. If it is possible is there a size limit for the file.

The file can not be created before hadoop job is created. I need to 
create and process the file inside map().

Thanks for your answer.

Ondrej Klimpera.

View raw message