hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Menzer <standar...@gmx.net>
Subject direct access to local file system for jobs possible?
Date Sun, 06 Apr 2008 13:42:39 GMT


Is it possible to run hadoop jobs in a way that input/output is not
read/written from/to hdfs but from/to local file system while using hadoop

I would like to avoid copying input and output data manually and would
prefer a more atomic approach to my current solution (copy input files - run
job - copy output files)

I need a practical possibility to directly compare the performance of the
hadoop cluster with a dedicated cluster (using MPI). Thus to be able to
compare them my hadoop job should also include reading and writing output to
local filesystem (a NFS mount) because this is also what my dedicated
MPI-Cluster does...

Also i think that accessing local fs directly (if possible) might be faster
than coping files in an extra step.

Thank you for any help, hint or information!

View this message in context: http://www.nabble.com/direct-access-to-local-file-system-for-jobs-possible--tp16524397p16524397.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

View raw message