accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devin Pinkston <devinfpinks...@gmail.com>
Subject Combiner Iterator
Date Wed, 18 Sep 2013 13:23:50 GMT
Hello,

I am trying to work with the example combiner iterator through java code
instead of the jar or shell.  My question is how do I pass in the
Iterator<Value> to the reduce method?  Usually I would create a Key
Value Iterator, but this requires an Iterator just over the Value, and
then the key to be passed in separately.  The reducer method comes from
the StatsCombiner class under examples/simple/combiner.  What I have
right now:

 Iterator<Map.Entry<Key, Value>> iterator = scan.iterator();
        Iterator<Value> iter;


        while (iterator.hasNext()) {
            Map.Entry<Key, Value> entry = iterator.next();
            iter = iterator.next().getValue();
            Key key = entry.getKey();
            Value value = entry.getValue();
            reduce(key, iter);
        }

How would I create the Iterator<Value>?  Every way I have tried has
led me to dead end or error at runtime.

Thanks!

Here is the reduce method from StatsCombiner.java:

 @Override
  public Value reduce(Key key, Iterator<Value> iter) {
    
    long min = Long.MAX_VALUE;
    long max = Long.MIN_VALUE;
    long sum = 0;
    long count = 0;
    
    while (iter.hasNext()) {
      String stats[] = iter.next().toString().split(",");
      
      if (stats.length == 1) {
        long val = Long.parseLong(stats[0], radix);
        min = Math.min(val, min);
        max = Math.max(val, max);
        sum += val;
        count += 1;
      } else {
        min = Math.min(Long.parseLong(stats[0], radix), min);
        max = Math.max(Long.parseLong(stats[1], radix), max);
        sum += Long.parseLong(stats[2], radix);
        count += Long.parseLong(stats[3], radix);
      }
    }
    
    String ret = Long.toString(min, radix) + "," + Long.toString(max,
radix) + "," + Long.toString(sum, radix) + "," + Long.toString(count,
radix);
    return new Value(ret.getBytes());
  }




Mime
View raw message