hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Udaya Lakshmi <udaya...@gmail.com>
Subject Logic for passing a directory using distributed cache is not working
Date Thu, 04 Feb 2010 11:45:21 GMT
I have used the following logic to send a directoy using distributedcache

PathFilter pf = new OutputLogFilter();
Path[]  fileList = FileUtil.stat2Paths(fs.listStatus(new Path(args[1]),pf));

        for(Path file : fileList)
        {
          DistributedCache.addCacheFile(file.toUri(), conf);
        }//for(Path file : fileList)

In configure of mapper class I have used

       fs = FileSystem.getLocal(new Configuration());
       localFiles = DistributedCache.getLocalCacheFiles(job);


In map I am using the following logic

       for(Path localFile : localFiles)
       {
         BufferedReader readBuffer = new BufferedReader(new
FileReader(localFile.toString()));
         while((str = readBuffer.readLine()) != null)
         {
            //some processing

         }//while(str = readBuffer.readLine() != null)
         readBuffer.close();
      }//for(Path localFile : localFiles)

But this logic is not working. Did I go wrong somewhere?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message