hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Something Something <mailinglist...@gmail.com>
Subject Custom Writable not working...
Date Sat, 14 Nov 2009 22:43:22 GMT
Hello,

I created a custom writable class called, TextPair, that's identical to the
one in Tom White's O'Reilly book (page 97).  The only difference is that the
compare function works differently.  In my case I want (1, 6) to compare
equally to (6,1), so I have something like this...

  @Override
  public int compareTo(TextPair tp) {
    int cmpFirst = first.compareTo(tp.first);
    int cmpSecond = second.compareTo(tp.second);

    if (cmpFirst == 0 && cmpSecond == 0) {
      System.out.println("Equal: (" + this.toString() + ") and " + "(" +
tp.toString() + ")");
      return 0;
    } else {
      int cmpFirstSecond = first.compareTo(tp.second);
      int cmpSecondFirst = second.compareTo(tp.first);
      if (cmpFirstSecond == 0 && cmpSecondFirst == 0) {
        System.out.println("Equal: (" + this.toString() + ") and " + "(" +
tp.toString() + ")");
        return 0;
      }
    }
    System.out.println("!!NOT Equal: (" + this.toString() + ") and " + "(" +
tp.toString() + ")");
    return -1;
  }


In my Mapper class I am emitting...

context.write(new TextPair(first, second), new IntWritable(1));


So I am expecting that when my Reducer is called the keys:

(1,4) & (4,1) would get combined.  This is NOT happening.  In fact, even
(1,4) are not getting combined.  In other words, the reducer is getting
called multiple times with keys (1,4)  (Makes sense?)

What am I doing wrong?  Also, I noticed that even after Reducer gets called
the TextPair.compareTo method keeps getting called.  By the way, my Reducer
is very simple.  It just adds counts (like the Word count program in the
tutorial.)

Please help.  Thanks.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message