incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Candler <B.Cand...@pobox.com>
Subject Reduce limitations
Date Sat, 25 Apr 2009 21:39:12 GMT
Has anyone found a clear definition of the limitations of reduce functions,
in terms of how large the reduced data may be?

I am considering using a reduce function for a low-ordinality value, to be
able to determine quickly what the unique values are, and how many of each.
That is, the final reduced value would be a small object of the form
   {"foo"=>50000, "bar"=>20000, "baz"=>30000}

(in this example there are 100,000 documents but only three unique values).

As I understand it, the final reduced object sits in the root node, so what
would be the practical limit on the size of this object? That in turn limits
the number of distinct values I can have.

I imagine this would be more efficient than a grouped reduce query (which
would have to re-reduce each key range separately), although I'm not sure by
how much.

Regards,

Brian.

  // implementation idea (completely untested)

  function(keys, values, rereduce) {
    if (rereduce) {
      result = values.shift();
      for(var i in values) {
        for(var j in values[i]) {
          result[j] = (result[j] || 0) + values[i][j];
        }
      }
    }
    else {
      result = {};
      for(var i in keys) {
        var key = keys[i];
        result[key] = (result[key] || 0) + 1;
      }
    }
  }


Mime
View raw message