hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lohit <lohit.vijayar...@yahoo.com>
Subject Re: Lots of files in a single hdfs directory?
Date Wed, 17 Sep 2008 21:04:21 GMT
Last when I tried to load an image with lots of files in same directory, it was like ten times
slow. This is to do with the data structures. My numbers were million though. Try to have
a directory structure.


On Sep 17, 2008, at 11:57 AM, Nathan Marz <nathan@rapleaf.com> wrote:

Hello all,

Is it bad to have a lot of files in a single HDFS directory (aka, on the order of hundreds
of thousands)? Or should we split our files into a directory structure of some sort?

Nathan Marz

View raw message