hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Update of "WriteHamaGraphFile" by thomasjungblut
Date Sun, 20 May 2012 20:23:00 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The "WriteHamaGraphFile" page has been changed by thomasjungblut:
http://wiki.apache.org/hama/WriteHamaGraphFile

New page:
This article is about how to bring your graph into a format the Graph module of Hama can read
and successfully let you run algorithms on it.

For this concrete example, the Google web graph from 2002 is used (http://snap.stanford.edu/data/web-Google.html),
we will give you a step by step guide from the download to a run of Pagerank on this dataset.
To run this example in your code, you will need the [[Guava Library|http://code.google.com/p/guava-libraries/]].


# to be extended
{{{
 Path txtPath = new Path(
        "/tmp/web-Google.txt");
 Path input = new Path(
        "/tmp/pagerankin.seq");
 HamaConfiguration conf = new HamaConfiguration(new Configuration());
 FileSystem fileSystem = FileSystem.get(conf);
 HashMultimap<Integer, Integer> map = HashMultimap.create();
 BufferedReader br = new BufferedReader(new InputStreamReader(
      fileSystem.open(txtPath)));
 String line = null;
 while ((line = br.readLine()) != null) {
   String[] split = line.split("\t");
   map.put(Integer.parseInt(split[0]), Integer.parseInt(split[1]));
 }
 Set<Entry<Integer, Collection<Integer>>> entries = map.asMap().entrySet();
 VertexWritable.CONFIGURATION = conf;
 SequenceFile.Writer writer = new SequenceFile.Writer(fileSystem, conf,
        input, VertexWritable.class, VertexArrayWritable.class);

    for (Entry<Integer, Collection<Integer>> entry : entries) {
      VertexWritable<IntWritable, DoubleWritable> key = new VertexWritable<IntWritable,
DoubleWritable>(
          new DoubleWritable(0.0d), new IntWritable(entry.getKey()),
          IntWritable.class, DoubleWritable.class);
      ArrayList<Integer> arrayList = new ArrayList<Integer>(entry.getValue());
      @SuppressWarnings("unchecked")
      VertexWritable<IntWritable, NullWritable>[] adjacents = new VertexWritable[entry
          .getValue().size()];
      for (int i = 0; i < adjacents.length; i++) {
        adjacents[i] = new VertexWritable<IntWritable, NullWritable>(
            NullWritable.get(), new IntWritable(arrayList.get(i)),
            IntWritable.class, NullWritable.class);
      }
      VertexArrayWritable val = new VertexArrayWritable();
      val.set(adjacents);
      writer.append(key, val);
    }
    writer.close();
}}}

Mime
View raw message