From user-return-9795-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Tue Apr 06 14:55:47 2010 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 34475 invoked from network); 6 Apr 2010 14:55:47 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 6 Apr 2010 14:55:47 -0000 Received: (qmail 82076 invoked by uid 500); 6 Apr 2010 14:55:46 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 81871 invoked by uid 500); 6 Apr 2010 14:55:46 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 81863 invoked by uid 99); 6 Apr 2010 14:55:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Apr 2010 14:55:46 +0000 X-ASF-Spam-Status: No, hits=-0.9 required=10.0 tests=AWL,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [80.244.253.218] (HELO mail.traeumt.net) (80.244.253.218) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Apr 2010 14:55:39 +0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.traeumt.net (Postfix) with ESMTP id 8AEA01B514 for ; Tue, 6 Apr 2010 16:55:17 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mail.g3th.net Received: from unknown by localhost (amavisd-new, unix socket) id X5YV4sZlwoAA for ; Tue, 6 Apr 2010 16:55:15 +0200 (CEST) Received: from [10.0.1.14] (62-220-4-154.berlikomm.net [62.220.4.154]) (authenticated) by mail.traeumt.net (amavisd-milter) (authenticated as web50m1); Tue, 6 Apr 2010 16:55:15 +0200 (CEST) (envelope-from ) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1078) Subject: Re: performance issues From: Jan Lehnardt In-Reply-To: <4BBB491D.6090803@julianmoritz.de> Date: Tue, 6 Apr 2010 16:55:15 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <84B4B5BF-5031-4893-AF68-93726B1B1E80@apache.org> References: <4BBA0C4A.9080304@julianmoritz.de> <4BBA315B.5010701@julianmoritz.de> <7FB459A8-6A46-4CC0-818B-AE60B554A673@gmail.com> <4BBA37C8.7010702@julianmoritz.de> <8F7808FC-5840-4142-A149-E2AA94AB0012@gmail.com> <4BBAEF76.40705@julianmoritz.de> <4BBB491D.6090803@julianmoritz.de> To: user@couchdb.apache.org X-Mailer: Apple Mail (2.1078) On 6 Apr 2010, at 16:45, Julian Moritz wrote: > Hi Jan, >=20 > Jan Lehnardt schrieb: >> On 6 Apr 2010, at 10:23, Julian Moritz wrote: >>=20 >>> Hi, >>>=20 >>> first of all: to make the keys of a view unique via reduce, i do it = as >>> it's desribed in the "definitive guide to couchdb". >>=20 >> try using "_sum" as your reduce function, it'll use an Erlang version = that >> will be faster than the JS version. "_sum" expects the emit() value = to be >> an integer. Just emit(doc.whatever, 1). >>=20 >=20 > my reduce-function does a `return True` atm. Would it really be a = speed-up? Not using JavaScript will be faster than using JavaScript. Feel free to = write your views in Erlang completely :) "_sum" gives you a convenient = middle-ground. > I just did what you are writing on p 185 in "the definitive guide". Yeah, I wrote that, but read on :) Cheers Jan -- >=20 > regards > Julian >=20 >> Cheers >> Jan >> -- >>=20 >>> J Chris Anderson schrieb: >>>> On Apr 5, 2010, at 12:19 PM, Julian Moritz wrote: >>>>=20 >>>>> But I need it for making my list unique. This is an important = feature >>>>> for my application. >>>> This is probably explains the slowness. When you do a group=3Dtrue = query, CouchDB has to run the reduce function once for each unique key = (serializing all the rows in the key to the JS process, and parsing the = results.) >>>>=20 >>>> I haven't tested this, but you might get better response throughput = by dropping the reduce function and using a _list which only sends one = row of output each time the key changes. This will avoid some additional = Erlang processing of the result. >>>>=20 >>>> Some documentation for _list is here: >>>>=20 >>>> http://books.couchdb.org/relax/design-documents/lists >>> Thank you, I will have a look on it. >>>=20 >>> Regards >>> Julian >>>=20 >>>>> Thanks, I'll think about how to set up a couchdb cluster and do = more >>>>> testing. >>>>>=20 >>>>> Regards >>>>> Julian >>>>>=20 >>>>>> Chris >>>>>>=20 >>>>>>=20 >>>>>>> Regards >>>>>>> Julian >>>>>>>=20 >>>>>>>> I've developed a (in my eyes) simple view. I have a wordlist = which does >>>>>>>> not contain unique words. I want to store it in a view, with = every word >>>>>>>> occurring once and ordered by random. Therefore I have a simple = view >>>>>>>> function: >>>>>>>>=20 >>>>>>>> function(doc){ >>>>>>>> emit([hash(doc.word), doc.word], null); >>>>>>>> } >>>>>>>>=20 >>>>>>>> and a simple reduce: >>>>>>>>=20 >>>>>>>> function(key, values, rereduce){ >>>>>>>> return true; >>>>>>>> } >>>>>>>>=20 >>>>>>>> calling that view with group=3Dtrue it does what I want. >>>>>>>>=20 >>>>>>>> When storing plenty of words to the database, one of my two cpu = cores is >>>>>>>> used completely by couchjs. >>>>>>>>=20 >>>>>>>> Isn't the view built using two (or all) cpu cores? I thought = (obviously >>>>>>>> I'm wrong) that it would be calculated in parallel and using a >>>>>>>> quadro-core (or more cores) would make storing faster. >>>>>>>>=20 >>>>>>>> Is there a solution for that? Should I use another = query-server? >>>>>>>>=20 >>>>>>>> Regards >>>>>>>> Julian >>>>>>>>=20 >>>>=20 >>=20 >>=20