hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Da Zheng <zhengda1...@gmail.com>
Subject Hadoop use direct I/O in Linux?
Date Sun, 02 Jan 2011 20:01:18 GMT

direct IO can make huge performance difference, especially when Atom processors
are used. but as far as I know, hadoop doesn't enable direct IO of Linux. Does
anyone know any unofficial versions were developed to use direct IO?

I googled it, and found FUSE provides an option for direct IO. If I use FUSE DFS
and enable direct IO, will I get what I want? i.e., when I write data to HDFS,
the data is written to the disk directly (no caching by any file systems)? or
this direct IO option only allows me to bypass the caching in FUSE and the data
is still cached by the underlying FS?


View raw message