Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6461CD948 for ; Fri, 16 Nov 2012 12:18:18 +0000 (UTC) Received: (qmail 2134 invoked by uid 500); 16 Nov 2012 12:18:15 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 1956 invoked by uid 500); 16 Nov 2012 12:18:14 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 1440 invoked by uid 99); 16 Nov 2012 12:18:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Nov 2012 12:18:13 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [93.93.131.52] (HELO haggis.mythic-beasts.com) (93.93.131.52) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Nov 2012 12:18:08 +0000 Received: from [2.96.107.255] (helo=[192.168.0.2]) by haggis.mythic-beasts.com with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1TZKrh-0007dF-Vm for dev@lucene.apache.org; Fri, 16 Nov 2012 12:17:46 +0000 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Apple Message framework v1283) Subject: Re: Merge information in segment files From: Alan Woodward In-Reply-To: Date: Fri, 16 Nov 2012 12:17:45 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: dev@lucene.apache.org X-Mailer: Apple Mail (2.1283) X-BlackCat-Spam-Score: -19 X-Mythic-Debug: Threshold = On = X-Virus-Checked: Checked by ClamAV on apache.org X-Old-Spam-Status: No, score=-2.0 Do you think it's worth promoting to a first-class API? Just a boolean = - isMerged(), or something. On 16 Nov 2012, at 12:11, Michael McCandless wrote: > We do actually record this, in the segments "diagnostics" field ... > but that format is something that can suddenly "change" (ie it's not > an API w/ back compat). >=20 > Mike McCandless >=20 > http://blog.mikemccandless.com >=20 > On Fri, Nov 16, 2012 at 7:01 AM, Alan Woodward > wrote: >> Hi all, >>=20 >> Is there any way of finding out if a segment is the result of a = merge, or if it's just new data? I can't find anything in SegmentInfo = that records this - if it isn't there, I'll open a JIRA. >>=20 >> Here's the use case: I need to reload ExternalFileField data when = segments are merged, as the internal docids will all have changed, = invalidating the EFF caches. However, new segments can just use default = values (the EFF is used to store things like click rates, which are all = zero for new data). At the moment, caches are refreshed after every = commit. But cache reloading is heavy - if we can restrict it to only = reload after a merge, then we save a lot of wasted CPU and IO cycles. >>=20 >> Thanks, >> Alan Woodward >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org >> For additional commands, e-mail: dev-help@lucene.apache.org >>=20 >=20 > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org > For additional commands, e-mail: dev-help@lucene.apache.org >=20 --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org