From java-user-return-63541-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Wed Jan 31 14:06:36 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id C3F7B180662 for ; Wed, 31 Jan 2018 14:06:36 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id B3833160C35; Wed, 31 Jan 2018 13:06:36 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 06DD8160C25 for ; Wed, 31 Jan 2018 14:06:35 +0100 (CET) Received: (qmail 41552 invoked by uid 500); 31 Jan 2018 13:06:34 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 41540 invoked by uid 99); 31 Jan 2018 13:06:34 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Jan 2018 13:06:33 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 2071619735B for ; Wed, 31 Jan 2018 13:06:33 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id woSyHlD9LtVe for ; Wed, 31 Jan 2018 13:06:28 +0000 (UTC) Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 38B54614F7 for ; Wed, 31 Jan 2018 13:03:24 +0000 (UTC) Received: by mail-wm0-f43.google.com with SMTP id t74so8003014wme.3 for ; Wed, 31 Jan 2018 05:03:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=a/oSDiHW486UB1vesTObT7TOFdP/6XtFvaUgQDtpkds=; b=LznZhSN7RQqTUIa1+S3E2tr34o1tCMfLKo2R5e63sImZXRnb8TudcDICtjh8neZU4I KFvtGDogGst/gMXeMD1zPDkWNAeAbm1N/dLVCSvQXFHm3MpMn9LSoOxLZQM1uOasEo8U one7wYeeulqDbNljyLVNfn0ljszPt7ifp9fX+ma4tA3nQaNUlBiFUoTaxMvDUmtuaaAx uYH4X+a5yLWIi/9BcxC/BmqwkFFjGtu30toA7kgFHSfRe4CmhuJpZUxx26F/bkhj7XHk BU6xHVOliaGiSgvMDApJ87r6tPpfUFLQyeG9tXDJMgvxIF2t5ruXxvR4tvQEkT+eKHyD oqIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=a/oSDiHW486UB1vesTObT7TOFdP/6XtFvaUgQDtpkds=; b=M6dX9ezADQi3DIQyKtwpfqHIdP+MkYfDmq6ZzyRZbN8gQnOSsUiF1dJzZak7ENrbT/ TOSTS5Z1v+zwiW568BU2vg2Xf33DUbDsbKTIpeKd8YMRgJ+rXOW7jxq0AGvTdrkpVEjP iE687iPEmvYyBpES/71UlDyEIpzu9dw7x9xk7wahpGHY1vEOYd5IwRp5/UgvD201UdBY TrXnh6yxMKJfqnaY+ZV5LCrMq0l/QSX7htUaDuLkxY0HTo2muZnQl+U1knDuBRTory/A eZXqR9QexUeLA4qoB1iz8k+MsSpc9GhxPVl8EMwGCZ2egyCy2qIZCAfAcR07R1HVSOeg w/Dw== X-Gm-Message-State: AKwxytcJY1mu8PQK52zl/lVSN6+3j/KTZtxHShrpDhLz0al3J13F7iqq uEa8KZJKO4ljzb7tquS8Z8AwQV/TP0Ms16qY+ck= X-Google-Smtp-Source: AH8x227LEttt4qUvzZiy75DERE78k5I+L/oI5RaRR6tG9q4YXb/c6TtyN6xSSmZZD+xg0pfAP7r8JYyt2imJfW2L7hc= X-Received: by 10.28.169.76 with SMTP id s73mr21193199wme.122.1517403803711; Wed, 31 Jan 2018 05:03:23 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Adrien Grand Date: Wed, 31 Jan 2018 13:03:13 +0000 Message-ID: Subject: Re: Increase search performance To: java-user@lucene.apache.org Content-Type: multipart/alternative; boundary="001a114ba852b37f050564121ac3" --001a114ba852b37f050564121ac3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Atul, Le mar. 30 janv. 2018 =C3=A0 16:24, Atul Bisaria a =C3=A9crit : > 1. Using ConstantScoreQuery so that scoring overhead is removed since > scoring is not required in my search use case. I also use a custom Sort > object which does not sort by score (see code below). > If you don't sort by score, then wrapping with a ConstantScoreQuery won't help as Lucene will figure out scores are not needed anyway. > 2. Using query cache > > > > My understanding is that query cache would cache query results and hence > lead to significant increase in performance. Is this understanding correc= t? > It depends what you mean by performance. If you are optimizing for worst-case latency, then the query cache might make things worse due to the fact that caching a query requires to visit all matches, while query execution can sometimes just skip over non-interesting matches (eg. in conjunctions). However if you are looking at improving throughput, then usually the default policy of the query cache of caching queries that look reused usually helps. > I am using Lucene version 5.4.1 where query cache seems to be enabled by > default (https://issues.apache.org/jira/browse/LUCENE-6784), but I am not > able to see any significant change in search performance. > > Here is the code I am testing with: > > > > DirectoryReader reader =3D DirectoryReader.open(directory); //using > MMapDirectory > > IndexSearcher searcher =3D new IndexSearcher(reader); > //IndexReader and IndexSearcher are created only once > > searcher.setQueryCachingPolicy(QueryCachingPolicy.ALWAYS_CACHE); > Don't do that, this will always cache all filters, which usually makes things slower for the reason mentioned above. I would rather advise that you use an instance of UsageTrackingQueryCachingPolicy. --001a114ba852b37f050564121ac3--