hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Optimizing Disk I/O - does HDFS do anything ?
Date Tue, 13 Nov 2012 20:30:01 GMT
How does HDFS deal with optimization of file streaming?  Do data nodes have
any optimizations at the disk level for dealing with fragmented files?  I
assume not, but just curious if this is at all in the works, or if there
are java-y ways of dealing with a long running set of files in an HDFS
cluster.  MAybe, for example, data nodes could log the amount of time spent
on I/O for certain files as a way of reporting wether or not
defragmentation needed to be run on  a particular node in a cluster.

Jay Vyas

View raw message