hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: how to force small files not to span over multiple nodes?
Date Wed, 01 Feb 2012 15:44:25 GMT
If the file size is less than a block size, then file isn't "spaning"
across nodes. Files are split at block size points, so your file is
essentially just one block here.

Also see http://search-hadoop.com/m/tGBgk1WFVAO1 for your block
location question. You can get the node list of replicas this way, but
not the explicit local paths.

On Wed, Feb 1, 2012 at 8:48 PM, Qiming He <qiming.he@openresearchinc.com> wrote:
> Hi all,
> Is there anyway (command) to determine the physical location of a file in
> HDFS to see it spans over multiple nodes? and any way to force a small file
> not to span over two nodes? assuming its size is smaller than default block
> size (e.g., 64MB).
> Thanks in advance
> -Qiming

Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about

View raw message