From solr-user-return-140759-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Sat Apr 21 18:52:34 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 6BE7018064C for ; Sat, 21 Apr 2018 18:52:34 +0200 (CEST) Received: (qmail 63675 invoked by uid 500); 21 Apr 2018 16:52:32 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 63659 invoked by uid 99); 21 Apr 2018 16:52:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 21 Apr 2018 16:52:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 22864C2AEE for ; Sat, 21 Apr 2018 16:52:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.102 X-Spam-Level: X-Spam-Status: No, score=-0.102 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=netlab1.net Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id COpIQXC2lKJs for ; Sat, 21 Apr 2018 16:52:29 +0000 (UTC) Received: from netlab1.net (netlab1.net [82.70.37.210]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 4FA325F3E3 for ; Sat, 21 Apr 2018 16:52:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by netlab1.net (Postfix) with ESMTP id 3E11A3026E893 for ; Sat, 21 Apr 2018 17:52:23 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=netlab1.net; h= content-transfer-encoding:content-language:content-type :content-type:in-reply-to:mime-version:user-agent:date:date :message-id:from:from:references:subject:subject:received :received; s=mail; t=1524329542; bh=oP3O+iEKA91u/Uz1QweAHNqob8vK E1k7zPdi9u+6QSc=; b=EAuFDugTTfKvp+SGzz3Kn2HQb3rXKI8VCYUG7UNtTdD/ Jl8NvDINHJVnttaP+8iRPadOjNb+srv4maChNzBMhjxNIxmUEhQXhONzLwSquyk5 fOB0xi5uQSLqZhVoziPyjs01+Csiw3L7c3kZb/7EUGbyBGjSBzmjsbHsD+IxDbo= X-Virus-Scanned: amavisd-new at netlab1.net Received: from netlab1.net ([127.0.0.1]) by localhost (netlab.netlab1.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id s5q81N9mXqQI for ; Sat, 21 Apr 2018 17:52:22 +0100 (BST) Received: from [82.70.37.214] (unknown [82.70.37.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by netlab1.net (Postfix) with ESMTPSA id 654903026E87A for ; Sat, 21 Apr 2018 17:52:22 +0100 (BST) Subject: Re: Loss of "Optimise" button To: solr-user@lucene.apache.org References: From: Joe Doupnik Message-ID: <34fc6ff9-512c-a678-9202-2b8b89370711@netlab1.net> Date: Sat, 21 Apr 2018 17:52:22 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: quoted-printable On 21/04/2018 17:25, Doug Turnbull wrote: > I haven=E2=80=99t tracked this change, but can you still optimize throu= gh the API? > > Here=E2=80=99s an example using update XML > > https://stackoverflow.com/questions/6954358/how-to-optimize-solr-index > > There are so many cases hitting =E2=80=9Coptimize=E2=80=9D causes a hug= e segment merge that > brings down a Solr cluster that I think I agree with the decision to re= move > such an inviting button. > > Doug > On Sat, Apr 21, 2018 at 8:08 AM Joe Doupnik wrote: --------- Doug, =C2=A0=C2=A0=C2=A0 Thanks for that feedback. Here are my thoughts on the= matter.=20 Removing deleted docs is often an irregular occurrence, such as say when=20 there is overlapping of incoming material with older input. We don' t=20 want to optimise often, least often in fact, but when the circumstances=20 do exist we want to do it expeditiously and wisely. That says a button=20 which we may use when needed, human judgement is employed, and thus=20 avoid automation which totally lacks that judgement and which leads to=20 often unnecessary system peak loads every day or so. =C2=A0=C2=A0=C2=A0 I don't want to search high and low for just the righ= t curl=20 command. The capability needs to be an intrinsic part of the management=20 facility design (say the GUI). =C2=A0=C2=A0=C2=A0 Clearly documenting the curl approach would be helpfu= l for many sites. =C2=A0=C2=A0=C2=A0 In my cases, I run my programs typically at night to = crawl this &=20 that, and do not wish to have to tend them daily. The situations of=20 overlap and thus deleted docs typically occurs when I am rebuilding the=20 background data sources, and that is when I am working on things and can=20 tend the overlaps. I do not want unnecessary peak loadings from automatio= n. =C2=A0=C2=A0=C2=A0 Thus reinstatement of the Optimize button would mean = humans could=20 again invoke it if and only when they thought appropriate. The admin GUI=20 would be the normal place to perform that, as it was previously. Lacking=20 that management facility would be a design shortcoming. =C2=A0=C2=A0=C2=A0 Thanks, =C2=A0=C2=A0=C2=A0 Joe D. >> In Solr v7.3.0 the ability to removed "deleted" docs from a core= by >> use of what until then was the Optmise button on the admin GUI has bee= n >> changed in an ungood way. That is, in the V7.3.0 Changes list, item SO= LR >> 7733 (quote remove "optmize from the UI, end quote). The result of tha= t >> is an apparent inability to remove piles of deleted docs, which amongs= t >> other things means wasting disk space. That is a marked step backward >> and is unhelpful for use of Solr in the field. As other comments in th= e >> now closed 7733 ticket explain, this is a user item whidh has impact o= n >> their site, and it ought to be an inherent feature of Solr. Consider a >> file system where complete deletes are forbidden, or your kitchen wher= e >> taking out the rubbish is denied. Hand waving about obscure auto-sizin= g >> notions will not suffice. Thus may I urge that the Optimse button and >> operation be returned to use, as it was until Solr v7.3.0. >> Thanks, >> Joe D. >>