couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manokaran K <manoka...@gmail.com>
Subject view generation speed
Date Tue, 11 May 2010 08:04:48 GMT
Hi,

Am testing an app where there are approx 400k docs (exam results). There are
only two views in the design_doc:

map:
function(doc) {
      if(doc.dtype === 'result'){
            var subject_total, scr, scores = doc.scores;
            var total = 0, len = scores.length; //scores.length is 6 to 12
            for(var i=0; i<len; i++){
                  scr = scores[i];
                  subject_total = scr.th + (scr.pr ? scr.pr : 0);
                  total += subject_total;
                  emit([scr.code, subject_total], 1);
            }
            emit(['to', total], 1);
      }
}

reduce:
function(keys, values, rereduce){
      return sum(values);
}

I use the above map-reduce to calculate the frequencies for each subject
score as well as the aggregate score.

My problem is that to it takes a looong time to generate the view. After two
hours (on a dual core 2.2GHz with 3GB RAM) only 300K docs have been
indexed!! Is that on par? Or is something wrong?

Sometime back JChris had mentioned using the erlang 'sum' call to speed up
things in response to another mail but I cannot locate that  now. Can
someone point me to it?

I have one other view in the design_doc:

map:
function(doc) {
      if(doc.dtype === 'subject_def'){   //only about 20-25 docs of this
type
            val = {};
            for(var prop in doc){
                  if(prop !== '_id' && prop !== '_rev' && prop !== 'dtype'){
                        val[prop] = doc[prop];
                  }
            }
            emit(doc._id, val);
      }
}

No reduce for this.


Thanks,
mano



-- 
Lord, give us the wisdom to utter words that are gentle and tender, for
tomorrow we may have to eat them.
   -Sen. Morris Udall

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message