hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Schneider <cschneiderpub...@gmail.com>
Subject How to tune fileSystem.listFiles("/", true) if you like walk though almost all files
Date Mon, 12 Aug 2013 19:33:18 GMT
Hi, is there a way to tune this?

I walk though the files with:

RemoteIterator<LocatedFileStatus> listFiles = fileSystem.listFiles(new
Path(uri), true);
while(listFiles.hasNext()) {
    listFiles.next();
};

I need to get some information about those files, therefore i like to scan
them all.

Is there any way to tune the listFiles.next() call. Like loading a bunch of
files, or multithreading it?

Best Regards,
Christian.

Mime
View raw message