Return-Path: Delivered-To: apmail-incubator-esme-dev-archive@minotaur.apache.org Received: (qmail 28389 invoked from network); 9 Dec 2009 10:01:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 9 Dec 2009 10:01:29 -0000 Received: (qmail 98517 invoked by uid 500); 9 Dec 2009 10:01:29 -0000 Delivered-To: apmail-incubator-esme-dev-archive@incubator.apache.org Received: (qmail 98470 invoked by uid 500); 9 Dec 2009 10:01:29 -0000 Mailing-List: contact esme-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: esme-dev@incubator.apache.org Delivered-To: mailing list esme-dev@incubator.apache.org Received: (qmail 98460 invoked by uid 99); 9 Dec 2009 10:01:29 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Dec 2009 10:01:29 +0000 X-ASF-Spam-Status: No, hits=-2.6 required=5.0 tests=BAYES_00 X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of vdichev@gmail.com designates 72.14.220.158 as permitted sender) Received: from [72.14.220.158] (HELO fg-out-1718.google.com) (72.14.220.158) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Dec 2009 10:01:27 +0000 Received: by fg-out-1718.google.com with SMTP id e12so9255fga.0 for ; Wed, 09 Dec 2009 02:01:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to :content-type; bh=pIHu0EIVscbQIImXRwFRIhvOMiiL7zy4OsExfvBTGdo=; b=deSpH1qMAh0h9iNwHWHPql9K3zKoFkHTZr3ypcZEYxaNxOCpYUVHi66qG7E36e2/3S v5NU6RvlhkTXga6R8zvm7u+27Og4Xn5NlU5I2hV0Bihe1FPbNFIATZZuAYj16koixXUr CwuPs1EmsajxJh1yy8igXxfl5eVRROBwvIvD8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=LcpPRQRoomSH/4yTc6BhCf4f2QStJDRGyhZohN9jiUNpi7sPXgsGsWvBz0YazGMxkx eBez1pbeX5P4hQzQqO+3IHUGdUn/znhn7nISuFPBBDEIyXfeqP/Eoa3v03Hk74dwe+aC wDsfrFmTnfkkR7bBRQ1Z6lz/IOGvJdAr9lCIg= MIME-Version: 1.0 Sender: vdichev@gmail.com Received: by 10.102.177.20 with SMTP id z20mr2709850mue.101.1260352865631; Wed, 09 Dec 2009 02:01:05 -0800 (PST) In-Reply-To: <771905290912090029s4073b3cbje5d16879cd1c19d5@mail.gmail.com> References: <771905290912071603y73cd4bbevd2ec7b2fd7284e54@mail.gmail.com> <771905290912080424s11a10f46xf7096d4ac7af72fc@mail.gmail.com> <771905290912090029s4073b3cbje5d16879cd1c19d5@mail.gmail.com> Date: Wed, 9 Dec 2009 12:01:05 +0200 X-Google-Sender-Auth: 773200951f90a10d Message-ID: Subject: Re: Performance update: Message size in memory From: Vassil Dichev To: esme-dev@incubator.apache.org Content-Type: text/plain; charset=ISO-8859-1 Markus, Good thinking- I would also suggest that we use a singleton stemmer for now (which would be KISS in our case). I also had some ideas to batch indexing, we could include it on our performance improvements list (after release 1.0?). For stemming it might be a better idea to have a stem cache rather than a pool of stemmers. Rendering does happen in the thread for the search, but I guess people won't notice a difference in performance normally. The other place where stemming happens is the word frequencies, but in the UI that is done as a comet snippet and pushed on demand, so there shouldn't be a noticeable delay. OK then, I will commit my changes today, which only involve moving stuff to the companion object. Of course, suggestions are welcome, but there's always room for improvement later on.