hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alberto Luengo Cabanillas <cabi...@gmail.com>
Subject READING A FILE AND PUSHING CONTENTS IN A STRUCTURE
Date Tue, 02 Feb 2010 23:45:55 GMT
Hi people! For the last two days I´m facing this problem: In the setup
function of  Reducer class, I´m trying to read a file (with a value per
line) located in HDFS and putting them into a Hashtable the following way:
@Override
protected void setup(Context cont) throws IOException{
        try {
            FileSystem hdfs = FileSystem.get(new Configuration());
            Path path = new Path(HDFS_REDUCER_CONFIGURATION_FILE);

            //Validation
            if (!hdfs.exists(path))
            {
                throw new IOException("File"
+HDFS_REDUCER_CONFIGURATION_FILE + "doesnt exist");
            }

            if (!hdfs.isFile(path))
            {
                throw new IOException("File"
+HDFS_REDUCER_CONFIGURATION_FILE + "doesnt exist");
            }

            FSDataInputStream dis = hdfs.open(path);
            String input="";

            while ((input = dis.readUTF()) != null) {
                String numPop = dis.readUTF();
                parameters.put("numPopulation", Integer.getInteger(numPop));
                String maxIter = dis.readUTF();
                parameters.put("maxIterations",
Integer.getInteger(maxIter));
            }
            dis.close();
        } catch (IOException ioe) {
            System.err.println("REDUCER: IOException reading Reducer
configuration file");
            System.err.println(ioe.toString());
        }
    }

But when I try to access the elements of the structure (parameters) I get
null values, caused-probably- because I´m not reading well those values.
What I´m doing wrong? Another way to do this?
Thanks a lot in advance.

PD: I forgot to mention that I´m working in pseudo-distributed mode.




-- 
Alberto

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message