From java-user-return-64522-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Wed Jul 10 14:52:43 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1FE0318064A for ; Wed, 10 Jul 2019 16:52:43 +0200 (CEST) Received: (qmail 89241 invoked by uid 500); 10 Jul 2019 14:52:40 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 89228 invoked by uid 99); 10 Jul 2019 14:52:40 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Jul 2019 14:52:40 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id F354E1A0DDD for ; Wed, 10 Jul 2019 14:52:39 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.094 X-Spam-Level: * X-Spam-Status: No, score=1.094 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_NO_HELO_DNS=1.294, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id y1erhNs4RZRZ for ; Wed, 10 Jul 2019 14:52:38 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.215.172; helo=mail-pg1-f172.google.com; envelope-from=jpountz@gmail.com; receiver= Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 84BECBC776 for ; Wed, 10 Jul 2019 14:52:38 +0000 (UTC) Received: by mail-pg1-f172.google.com with SMTP id q4so1378954pgj.8 for ; Wed, 10 Jul 2019 07:52:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=hhcUFPbDD4OzvoET7y0oVz24OTtNayb3F0OjUDWwMYw=; b=k779H64z8CUIuJEhHmtUHynWDIyxszqSBnO1SiMFqDnAH7dYBQMd0c2O8vmN9yiGKq EFHRoQzxGyAj1AGZM4ucuzG9/dw8QfXcrixSSF6rsSR1maSpM7k2LxdOhyxuOAc6Ec5b zlBhQQQ8n3N57IFyTvp8net0C3yCWgksagWhzODlXiyR3QAvWYhNYKy9oQ0C67MTQhZJ 2oQNU9f/PUarN/4RAchdDM0/B8d+ZV2fDp+LntWCHCBnHRaqPYXWJNvnKdLcVCLRLybs NidaOu0QAChdjn0G+MptupbywDsZm1T/DwqGKfii/iaCHB/C4LJSoyq48L4ztwhe9cMi wTdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=hhcUFPbDD4OzvoET7y0oVz24OTtNayb3F0OjUDWwMYw=; b=Qm5iu6XFMcnlf/8d1mkW5T/XFIk3z/DLeALrAL2qUnxTM0EVM/I1a5/UqEGuI8ylYb kVnrErokw/O4pkoHHcQekq9aHbny5C2Re1qNHtsfdsS/nBRINgEzGaiQz6TvRfCOrcdZ hdpTaMzlhO+AS2WdTew9RKWds4TSM5nWi6RRf+6foGwmvPaWogObnJorv3JlPXguetIt E0pL+AzOas779Lrz1OPaB9tl92Fj4/5q0vaZYfCDx3f3AyiKHhpzvkzNQs1iBKmopXDc Y/tjJbURFuh4s7xcGjkrene8d3LkfFRgBpl0PRt7KpLr2PVhxExNc1/6Av5W3vCeadVy miNw== X-Gm-Message-State: APjAAAXxLPjYkhkjeEXx9FtFCjIanf8UsCvEiJk9BvWSs+QQBk1cm6A2 7XqXPc4vUqqQfBgiWyO07UiUPUI+eKGiKi4woNzO/SwH X-Google-Smtp-Source: APXvYqygtgO8EeMcVdftjbLYML+q9WioWYYE959N7yZHFdA7bQS9/PllO7ZxWsIdlrdlITbiicVKKmAWdyYFa6B6+AE= X-Received: by 2002:a63:fd0d:: with SMTP id d13mr38684983pgh.423.1562770351740; Wed, 10 Jul 2019 07:52:31 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Adrien Grand Date: Wed, 10 Jul 2019 16:52:20 +0200 Message-ID: Subject: Re: Impact and WAND To: Lucene Users Mailing List Cc: "Wu,Yunfeng" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable To clarify, the scoring process is not accelerated because we terminate early but because we can skip low-scoring matches (there might be competitive hits at the very end of the index). CompetitiveImpactAccumulator is indeed related to WAND. It helps store the maximum score impacts per block of documents in postings lists. Then this information is leveraged by block-max WAND in order to skip low-scoring blocks. This does indeed help avoid reading norms, but also document IDs and term frequencies. On Wed, Jul 10, 2019 at 4:10 PM Wu,Yunfeng wrote: > > Hi, > > We discuss some topic from https://github.com/apache/lucene-solr/pull/59= 5. As Atri Sharma propose discuss with the java dev list. > > > Impact `frequency ` and `norm ` just to accelerate the `score process` = which `terminate early`. > > In impact mode, `CompetitiveImpactAccumulator` will record (freq, norm) p= air , would stored at index level. Also I noted `CompetitiveImpactAccumulat= or` commented with `This class accumulates the (freq, norm) pairs that may = produce competitive scores`, maybe related to `WAND`? > > > The norm value which produced or consumed by `Lucene80NormsFormat`. > > In this ` Impact way`, we can avoid read norms from `Lucene80NormsProduce= r` that may generate the extra IO? =EF=BC=88 the norm value Lucene stored = twice.=EF=BC=89and take full advantage of the WAND method? --=20 Adrien --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org