From solr-user-return-141475-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Wed May 30 09:25:33 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 47D5F18063B for ; Wed, 30 May 2018 09:25:32 +0200 (CEST) Received: (qmail 74124 invoked by uid 500); 30 May 2018 07:25:30 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 74107 invoked by uid 99); 30 May 2018 07:25:29 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 May 2018 07:25:29 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 20EAD1A2904 for ; Wed, 30 May 2018 07:25:29 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.898 X-Spam-Level: * X-Spam-Status: No, score=1.898 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=vitec.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id f_MqQqDt0RpP for ; Wed, 30 May 2018 07:25:25 +0000 (UTC) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0040.outbound.protection.outlook.com [104.47.0.40]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 5740D5F230 for ; Wed, 30 May 2018 07:25:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vitec.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5yuZqt1+PKhOomuPTePXXATLTCkB6hnWeL/QaCTzEJ4=; b=SkfYy5X0A0k3Ff6cGodA0jj1XsiKcLG9gcP6hKQup3zcuxddDZ9aiS5nproqqifYqDtvp8SNyume3EvZ/pRw6nSakCOJJg5ldwZVepgK5/yUbufVjbgumimdZmHFn8H9YiGS+ka2W+TMUA5t2HhtJL9usUEe3y47NNrIuvwsuc8= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=jens.viebig@vitec.com; Received: from [10.1.1.42] (213.158.103.243) by AM6PR0502MB3797.eurprd05.prod.outlook.com (2603:10a6:209:a::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.797.11; Wed, 30 May 2018 07:25:15 +0000 Subject: Re: Impact/Performance of maxDistErr To: solr-user@lucene.apache.org References: From: Jens Viebig Message-ID: <4d500993-2f9d-292c-469f-283901982acc@vitec.com> Date: Wed, 30 May 2018 09:27:49 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------B5DB6867702107CCCBAC68E5" Content-Language: de-DE X-Originating-IP: [213.158.103.243] X-ClientProxiedBy: AM6PR0502CA0009.eurprd05.prod.outlook.com (2603:10a6:209:1::22) To AM6PR0502MB3797.eurprd05.prod.outlook.com (2603:10a6:209:a::26) X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(49563074)(7193020);SRVR:AM6PR0502MB3797; X-Microsoft-Exchange-Diagnostics: 1;AM6PR0502MB3797;3:HxFINXxMv5ROgMQtt9o5oAYFGJKAKSAwM64wXWluQFrEcHh1Aw24/ptvpKLfvYs8i9uMMwArUrvdCNQjLcDKSXVzGIA3QyA+TFEMovXQsoIvRXcQclviUWqNP/rA1CLK9/4L3wYWu72RAfbTFFwIMID++x49jv0DvioW0mFrFGCZxnGtaaby49kCEiFfhVs6tGVY7VpIZyIsPzJDzp+/P51JTz3kfesRVtulGWVqWQGugexCDrsPkuemj7+NrMNB;25:YCCoO9n3I/UgbJeApKzN/P8evtWHJNDF2d+BXxu/tItm5iRpVH7Gjggemgpw1wyAt/w5C522ieBZ5eZZajE8bIV/Zea2R7dCkNlKLsAsnFDmq2XwBTXjHb8nKRSWxdaWDXsg2JlsGo8WivO6+12DK0tVWs3ajuACQ/xxq04Qw39U3V+/fDd/r2BlV1XmbNYpWkel6XHizzS1glLR5L+RJl4JqLyUYaCQXJ6hdwDPLfel/EtsadTEIFyIXaebGL3ma4FlqhVi1HsqC7jjKH2ykdwnmDfRBjUbvKa3LMs/zWVxT17onl8ZMdZOH0OAIum8HQOn/4jVL5+dZnmWJPcp4g==;31:adaeS1wceBldX/iGwlSmcnsJgFZ9cjofrRzNDi1kbQgOZijV/TVk1gL51Dr2ZsIGlu3bOEBXf/iAauEl+tTWQuD1HrZcFVfDa06uWHPJYSFPAOLB5Q5xekANEVuNyM7mvsgcdgWVpnqOMmz2tgpnEijw6KNOPTbr+TX+/zqbIhKiEDaJQ1NqPWp1t8gvwvqjffPIwe1nkfIG9hRmqmpF+qfwloWaLMvSfc8JTHAe6YA= X-MS-TrafficTypeDiagnostic: AM6PR0502MB3797: X-Microsoft-Exchange-Diagnostics: 1;AM6PR0502MB3797;20:+xQF2yjXFIcHusO1LlIKrSsMc3syrLQJtQEQ6So3PLvHo+D3Z06BLDMbL2hOXM/05CVul7kPPUXWDb9nowAmITYwH/xOMZaw6tbCnJwPUkmLl/tWDfd8Ws5LE5I4v+MBx2iLU+klTOp3J5EXu+tFTTk3VSQscVEfo+ijAgW61AlUrgrpN9VTVUUbeNO1i6wrKaimsrx6VRbgwGrjB7lh9T6+rA9oWJbsEo7cnlNqko0hG20zjNrJjfHYESl1N02D0WiAhcNuU5fK8tONdM6fZQ0kIvvPyFKOathsp6+lha9uSyG4OzoRIGqmTWNAowBerNRx1T894Wn8Ph2NMTw9BvGZh71evkBTxwaxH/67ivrezyv8IT2yk4ItdXshd8r0MxEAzv/CfbZvVa0fG+2wGQ6zQITMK+QRg7v3gd7s2/hdXme3GBE6/Z9CfdD5VlAJ1tPPdV6ndfWHjOIvSPnsRDbW/DwDZpr2b9+coiwBQAKBUEVUzj9xhTwTkZ5Ibpe5;4:zxs1E1wcDGjnYxQAZ03cVurSyplJ3qmlzabHrGg0VhDE80wbaweKhrcrsauYy1Q6CgxwpNAOHkpC+txHs+DgROO9MhV3s/310lEQ+5mytKgFMQVYCSJDf4yofSoA7yBMiJ4QZrdjlYqr8hThraiYQIBYlGsam4PtInbGx4bV3EM76dGXMqgY2+3KZ8yUSr7Qne4uD5ZQodh6TV/euXg7luTrwHah5Rg9wfNQJfN5zkZbr0p0vGC/gdjjlWmmaVubRbM9qW12hDduQghNHEwVZD2AiUZBWeq/pEeGL0gY5I2xl1Z10pCpCGX3uNA3fo7Dlw0T3PvVmIcToXCxXvBEDYw2HjknR97UshJTc4RAoIoZGCsu6ckGIj3m0B21bB1mb/gwpxa1VQqjVnMiM0lAKUHYPUcBVR4VaPwdYDjOFIs= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(209352067349851)(116415991822766)(85827821059158)(81160342030619); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(102415395)(6040522)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231254)(944501410)(52105095)(149027)(150027)(6041310)(20161123562045)(20161123564045)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016);SRVR:AM6PR0502MB3797;BCL:0;PCL:0;RULEID:;SRVR:AM6PR0502MB3797; X-Forefront-PRVS: 0688BF9B46 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6049001)(39840400004)(366004)(39380400002)(396003)(376002)(346002)(51914003)(199004)(189003)(25786009)(54896002)(64126003)(6306002)(76176011)(52116002)(6116002)(5660300001)(386003)(3260700006)(6246003)(33964004)(26005)(11346002)(446003)(53936002)(18717965001)(2906002)(3846002)(2616005)(956004)(16526019)(66066001)(59450400001)(186003)(68736007)(53386004)(58126008)(3480700004)(53376002)(84326002)(476003)(1720100001)(81166006)(44832011)(65956001)(53546011)(8676002)(486006)(8936002)(37036004)(77096007)(65806001)(606006)(45080400002)(16586007)(81156014)(229853002)(16576012)(2351001)(2361001)(97736004)(316002)(6916009)(478600001)(236005)(31686004)(36756003)(7736002)(966005)(65826007)(106356001)(6666003)(105586002)(733005)(6486002)(31696002)(54556002)(86362001)(19627235001);DIR:OUT;SFP:1101;SCL:1;SRVR:AM6PR0502MB3797;H:[10.1.1.42];FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: vitec.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM6PR0502MB3797;23:kzBgN397Pulg+7vjU3HK3IK+L8JOI3tRXN9IQMY?= =?us-ascii?Q?m8qoWNnfvLI2ECVmrrQ0Pu161FJxxQuZ/5BK/j9xIwn8+qyDSioXSS7hd509?= =?us-ascii?Q?jFVZAvCZ/nKI1yA4Q4qRvhSYBMvqmT67t+nfYNDnaUDRzc0Owr3maKVUO0gm?= =?us-ascii?Q?YpQeBM/DzM61DM106btSU58AiZHqaFXfky3upcjkU0M8brC66Mx2rmbozUyH?= =?us-ascii?Q?Kg6sP0F4mPMKCuLoqVIMGi1l8ehhT/j3ZRqKd/SQyrXLjyt+FUoineD/M4v5?= =?us-ascii?Q?eSdTauQV2bdUM8Jz2WYjdNhCK9rJallUKEpokx7jVVsa2s4J9q0iQgOOOkeF?= =?us-ascii?Q?BZo77/SeQrAv9h4vZxmLDzwS0o8Xpd/ckcyjbrqodM3n3CzYFYCCWrhUTf3C?= =?us-ascii?Q?Zhg+psiXcpiXLaXIWVVXT6QG5xdLxyR86IrjDtsVlLuroSYpiiuTGzQYwTJJ?= =?us-ascii?Q?whbvke/R2g2p+iCnVNReHZa6Q545bf8Qkgsi1copH39Yw5jXrbP8ZLyYiiDo?= =?us-ascii?Q?mxv/aO9nIp3W3ulbHzUcoKdvWAf0+b2DDSvECOsIgaxpRuhrpjg4yqzyVYSm?= =?us-ascii?Q?5ceeosmlZqjZtSFlpx4PdSHtCW4c7hceyulVZxz2NB+eZCHOIX9wjmxvDDt5?= =?us-ascii?Q?FcC239v6+Fg0XiodycKA0jH84hb27334aZRAVWz4opILut9TonoUhBOYtINa?= =?us-ascii?Q?j8s9RwKEvXJXKvGHHKF4GTDsVmrMoSlmo4Wi+kUH8Agki8dExXaKrMD5TDSP?= =?us-ascii?Q?1GlfshITIDD0B8kA9s9Fu6/7mi2gcGSfvnH2jKuuPDr9K+Q6r3Gh+JhG6pHk?= =?us-ascii?Q?I+QxBHqEyE327GOnW44eBcTUzcjLG3QCHNOV4WG8D00ia4ofj3oVPD5ciPAS?= =?us-ascii?Q?SZXVh4gGLxZ3AMuWQIJUeTEtbE/7knOpMfPGs3qRIOFol7OdpB32whus+rYw?= =?us-ascii?Q?CBtSZwG2ysL0rfoEXa4EWE95ptqrgeIDV2L5MIniCvzpQzmYGoMX01fqrsAv?= =?us-ascii?Q?jjwHZJzEgW6RCQvIx5JVefRn3+2A3/rRmOIY4+XbIwm3cRnnPU+4qWPZ3ULB?= =?us-ascii?Q?Q4nlm7VZ8LiRTPH+vkRf0i2Uubz4Om0i/6uYhCGjrtEvQxLvtE1cfTW+y4zS?= =?us-ascii?Q?g1js50BKfBvhMlbh3bdCvNKs72KElXeDv5cRtPpZBaQxmtp9fH4LWc0z1eyC?= =?us-ascii?Q?OuJAWjFlU3EnJQC9CaUkNYmFbrSE7t7anGUyQBYIDnUXUQmJUOG1paSqrdy0?= =?us-ascii?Q?0CBEn2BVVNXyAaVG6G8mC1PcQciQo0M0B3S54XtACL3W6L7hHKGON8MK2DN7?= =?us-ascii?Q?T0Jef+O5lOqzgpckO2fqj7cpXe5rch5fZrv//Bkn1eWbluQKphLAPQ78WYiq?= =?us-ascii?Q?7odcBynHpdCN9GDtfHwegZVhZM8Z/E3X2JZ+pySg5ShJQQOJ3Gd67cS07LpA?= =?us-ascii?Q?X0c+FPPfCV7rqu8ruWQQCU7h5hGmadcMhPml0DVy0Glq83lSfKBrYMY6yFHD?= =?us-ascii?Q?PKWAH3O3vG6MGakkrsUBRxIxYYAfMqgaPOhywQ1W6SFBrCai2J9pug9aKxFX?= =?us-ascii?Q?Dxovt5QByO8YiYxR9GVLrdNJ2hjQIca5p93jwqzd3sA1lrUSBPxFRNNJfhW6?= =?us-ascii?Q?h91SHeEUZf6TE5Uh4uk4oGyu+wA5bpUucPUvqIEkzFnG4DYmArl4kAtveSA/?= =?us-ascii?Q?QAz8v2RaVSmKfPxPaOpZ98s9YYUuyNVYWUdaf6o4bPY7nNGCCrGWxJmIyEDH?= =?us-ascii?Q?vmo2VTrnALQLjDQmbJvE+r099p+T4+JPqAESM8DiuLtDz/+YhZ2E=3D?= X-Microsoft-Antispam-Message-Info: ++meBWIhWp3nB66eLPOZ5DwLevtGtu6q0hHIinOjAhLCoXBVjX2HbY7kRr90JuC2HOQjR2z4c/fWqBqw/7ba9zdrnUpD0YO6YNG8NpzLJw7dKCmreVBSxXVFBkS2fh2NEeFSjkSAlywKX5mAb6x6tkm+8vHibpiiKrFMB3OsN1uPEThUOaDQfLYUO3ZdcETV X-Microsoft-Exchange-Diagnostics: 1;AM6PR0502MB3797;6:z9hPyLmRtEmlAOh8CqRZleZxKP3onU+ayChfVfhdDm/2hkote2obUUVXaDa9NvPggonZupZg1dSzpeCv/Src9yYbXR7fadJPCqclXQuL1vJfLhBUq98N7be3rx+AMxB2RzEEYxvu7uXrqv7SB1qBAKT4Df6y4cMpnjQsVgNApKoVpIrhFABilwcFS1qa9/o8dBO4rbmb5I8LCmc+RTCpjAf7uF9qYoVCFMyzok67cJBT/JPfsxtSCrEdZXrsHMVYBFpwnwNrhs3PAbBEV/3jD/LaDn8NXGWTgI6dlPpioWYAVPOMhBlynH0/DXYsOVAB6QJMEgx712/Nj6Nt7VVZ0LyF4t4cFqlIOAzLetlntwGBaUNTfio/nl6j4M2xqiR4WZQvxtWgpY1x8eXs8e4OkUc0kE1xZ1mIZtOMx7QYkf04+U2yDNqe8hLDagWVlTl00lztGhHukrr2aBSupcOh6g==;5:z3Y8Yq7GylAAejTCuFOvEG869cZX2YTV9x1Ucutpawgtd22uyATT2hVTWxuEsr1WXvOfMCb51pZuMD1LLFBFaFMaMLmLl+aCqFT3laQ78PDnJ54ve/YNBkZNiYIdvAYuTi/WH1E+midvfUTc+a+ogxqb8Ca7gmiMeSSMSSadXK8=;24:O/Ioh10BbSysbzOvdS/shntteUbViH/p0wF9yvG4pXVfmq2NT1eIAyXJMCzLh9y0FGmj9Rb4Thhs7slSME3wqQ9uvpsyiLuBq7hKvHZbB7o= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;AM6PR0502MB3797;7:cwyOXHoXaKj6uwVKB4BCkhkLfWMvICQ+Ic3hnAjlBgHzZWgRRxxtjPM9SFIgQfMQAdu4ZvAg7JQWgdlzv4xbm0nfLEFVU4t9UxZWA4/rq+V/rxg0QEHhf6n5yvj9XdcBlV0V0mkMILqRnq7huW52ZYa1QCi4XhzgLrHIff7iWZDfpTpxUANUtLjW6kDM0BxJl5I2p6DKYYk8+hihncvlFsxndcjoU2llO2rMSuBcx/Pj0eHKpM66yaALkem7xBs3;20:ivgnZqvTcCbeB4H0++Jrz+gSc7Dz4BL9MBJI5T9M/PzLYIY9eb6phaIEKVBiedY7RoW07NyYOrgPySVnjUXNeV2XJddpL0RAUUDeO5r3Ee/Jk0XaXbxgUWeq/3JgMp1Uo7kvvIh7Ut/kS/V/K1B0xmlrflczCcnxC2Akt6N0xIeHfwojsQnyHUNnaILETrIANtsJYalXH2/zoNcmQ9jNKu0lSYnREoI6JRy3sE3cC+bURePN2BNP5h1qFWJVpkLQ X-MS-Office365-Filtering-Correlation-Id: 1cefd992-e207-4017-8831-08d5c5fe7d66 X-OriginatorOrg: vitec.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2018 07:25:15.4811 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1cefd992-e207-4017-8831-08d5c5fe7d66 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 417d3214-b611-4fca-a1d5-e4446a9f50f4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0502MB3797 --------------B5DB6867702107CCCBAC68E5 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Thanks for the detailed answer David, that helps a lot to understand! Best Regards Jens P.S. Currently the only search we are doing on the polygon is Contains(POINT(x,y)) Am 29.05.2018 um 13:30 schrieb David Smiley: > Hello Jens, > With solr.RptWithGeometrySpatialField, you always get an accurate > result thanks to the "WithGeometry" part.  The "Rpt" part is a grid > index, and most of the parameters pertain to that.  maxDistErr > controls the highest resolution grid.  No shape will be indexed to > higher resolutions than this, though may be courser resolutions > dependent on distErrPct.  The configuration you chose initially (that > turned out to be slow for you) was a meter, and then you changed it to > a kilometer and got fast indexing results.  I figure the size of your > indexed shapes are on average a kilometer in size (give or take an > order of magnitude).  It's hard to guess how your query shapes compare > to your indexed shapes as there are multiple possibilities that could > yield similar query performance when changing maxDistErr so much. > > The bottom line is that you should dial up maxDistErr as much as you > can get away with it -- which is as long as query performance is good. > So you did the right thing :-).  That number will probably be a > distance somewhat less than the average indexed shape diameter, or > average query shape diameter, whichever is greater.  Perhaps 1/10th > smaller; if I had to pick.  The default setting, I think a meter, is > probably not a good default for this field type. > > Note you could also try increasing distErrPct some, maybe to as much > as .25, though I wouldn't go much higher., as it may yield gridded > shapes that are so course as to not have interior cells.  Depending on > what your query shapes typically look like and indexed shapes relative > to each other, that may be significant or may not be.  If the indexed > shapes are often much larger than your query shape then it's significant. > > ~ David > > On Fri, May 25, 2018 at 6:59 AM Jens Viebig > wrote: > > Hello, > > we are indexing a polygon with 4 points (non-rectangular, > field-of-view of a camera) in a RptWithGeometrySpatialField > alongside some more fields, to perform searches that check if a > point is within this polygon > > We started using the default configuration found in several > examples online: > > class="solr.RptWithGeometrySpatialField" > spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory" >            geo="true" distErrPct="0.15" maxDistErr="0.001" > distanceUnits="kilometers" /> > > We discovered that with this setting the indexing (soft commit) > speed is very slow > For 10000 documents it takes several minutes to finish the commit > > If we disable this field, indexing+soft commit is only 3 seconds > for 10000 docs, > if we set maxDistErr to 1, indexing speed is at around 5 seconds, > so a huge performance gain against the several minutes we had before > > I tried to find out via the documentation whats the impact of > "maxDistErr" on search results but didn't quite find an in-depth > explanation > From our tests we did, the search results still seem to be very > accurate even if the covered space of the polygon is less then 1km > and search speed did not suffer. > > So i would love to learn more about the differences on having > maxDistErr="0.001" vs maxDistErr="1" on a > RptWithGeometrySpatialField and what problems could we run into > with the bigger value > > Thanks > Jens > > *** > > > > * > > *Jens Viebig*** > > Software Development > > MAM Products > > > T. +49-(0)4307-8358-0 > > E. jens.viebig@vitec.com > > _http://www.vitec.com_____ > > __ > > VITEC_logo_for_email_signature__ > > -- > > VITEC GmbH, 24223 Schwentinental > > Geschäftsführer/Managing Director: Philippe Wetzel > HRB Plön 1584 / Steuernummer: 1929705211 / VATnumber: DE134878603 > > -- > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: > http://www.solrenterprisesearchserver.com -- Fwd: so, nun aber... *** * *Jens Viebig*** Software Development MAM Products T. +49-(0)4307-8358-0 E. jens.viebig@vitec.com _http://www.vitec.com_____ __ VITEC_logo_for_email_signature__ -- VITEC GmbH, 24223 Schwentinental Geschäftsführer/Managing Director: Philippe Wetzel HRB Plön 1584 / Steuernummer: 1929705211 / VATnumber: DE134878603 --------------B5DB6867702107CCCBAC68E5 Content-Type: multipart/related; boundary="------------E63235C68732C863C955B2C7" --------------E63235C68732C863C955B2C7 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

Thanks for the detailed answer David, that helps a lot to understand!

Best Regards
Jens

P.S. Currently the only search we are doing on the polygon is Contains(POINT(x,y))

Am 29.05.2018 um 13:30 schrieb David Smiley:
Hello Jens,
With solr.RptWithGeometrySpatialField, you always get an accurate result thanks to the "WithGeometry" part.  The "Rpt" part is a grid index, and most of the parameters pertain to that.  maxDistErr controls the highest resolution grid.  No shape will be indexed to higher resolutions than this, though may be courser resolutions dependent on distErrPct.  The configuration you chose initially (that turned out to be slow for you) was a meter, and then you changed it to a kilometer and got fast indexing results.  I figure the size of your indexed shapes are on average a kilometer in size (give or take an order of magnitude).  It's hard to guess how your query shapes compare to your indexed shapes as there are multiple possibilities that could yield similar query performance when changing maxDistErr so much.

The bottom line is that you should dial up maxDistErr as much as you can get away with it -- which is as long as query performance is good.  So you did the right thing :-).  That number will probably be a distance somewhat less than the average indexed shape diameter, or average query shape diameter, whichever is greater.  Perhaps 1/10th smaller; if I had to pick.  The default setting, I think a meter, is probably not a good default for this field type.

Note you could also try increasing distErrPct some, maybe to as much as .25, though I wouldn't go much higher., as it may yield gridded shapes that are so course as to not have interior cells.  Depending on what your query shapes typically look like and indexed shapes relative to each other, that may be significant or may not be.  If the indexed shapes are often much larger than your query shape then it's significant.

~ David

On Fri, May 25, 2018 at 6:59 AM Jens Viebig <jens.viebig@vitec.com> wrote:

Hello,

we are indexing a polygon with 4 points (non-rectangular, field-of-view of a camera) in a RptWithGeometrySpatialField alongside some more fields, to perform searches that check if a point is within this polygon

We started using the default configuration found in several examples online:

<fieldType name="location_grpt" class="solr.RptWithGeometrySpatialField"
           spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"
           geo="true" distErrPct="0.15" maxDistErr="0.001" distanceUnits="kilometers" />

We discovered that with this setting the indexing (soft commit) speed is very slow
For 10000 documents it takes several minutes to finish the commit

If we disable this field, indexing+soft commit is only 3 seconds for 10000 docs,
if we set maxDistErr to 1, indexing speed is at around 5 seconds, so a huge performance gain against the several minutes we had before

I tried to find out via the documentation whats the impact of "maxDistErr" on search results but didn't quite find an in-depth explanation
From our tests we did, the search results still seem to be very accurate even if the covered space of the polygon is less then 1km and search speed did not suffer.

So i would love to learn more about the differences on having maxDistErr="0.001" vs maxDistErr="1" on a RptWithGeometrySpatialField and what problems could we run into with the bigger value

Thanks
Jens

Jens Viebig

Software Development

MAM Products


T. +49-(0)4307-8358-0

E. jens.viebig@vitec.com

http://www.vitec.com

 

VITEC_logo_for_email_signature

 

--

VITEC GmbH, 24223 Schwentinental

Geschäftsführer/Managing Director: Philippe Wetzel
HRB Plön 1584 / Steuernummer: 1929705211 / VATnumber: DE134878603

 

--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker

--
Fwd: so, nun aber...


Jens Viebig

Software Development

MAM Products


T. +49-(0)4307-8358-0

E. jens.viebig@vitec.com

http://www.vitec.com

 

VITEC_logo_for_email_signature

 

--

VITEC GmbH, 24223 Schwentinental

Geschäftsführer/Managing Director: Philippe Wetzel
HRB Plön 1584 / Steuernummer: 1929705211 / VATnumber: DE134878603

 

--------------E63235C68732C863C955B2C7 Content-Type: image/gif; name="VITEC_logo_for_email_signature.gif" Content-Transfer-Encoding: base64 Content-ID: Content-Disposition: inline; filename="VITEC_logo_for_email_signature.gif" R0lGODlhogAzANUAAEBZjp+rxmB1oe/x9hAvcd/j7CA9e6+50FBnlzBLhM/V43CDqo+dvf/d f7+VGv/77//AEP/JMP/mn//VYP/yz//33//qr7/Dyf/EIP/ij//ZcP/NQP/RUP/uv7+aKv+8 AL/H2X+QswAhaP///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAAAAAAALAAA AACiADMAAAb/wJFwSCwaj8ikcslsOp/QqHQ6AIEOhaR1a31yrQph4Rsejb/o9Fesbn+zyQJD AKjXF4HBdDk4hBAJIoIiAkMPDUUAg4MHTQeLgoUjIZAAQpSQmZqQl5uekCFIAYGfAnB7QwcL pJmSIxQREkUBrU0CmWWYg5aTn76Dnb+eoUUFisIixFMKAgSlQxQQEEYDmQRNzosGQ7qCvN3I mcHhoEUK2eGuT6O/rtEfE0cImY1Kj5AL3JXj5Jy9/YuUjTgHkNA6A8IQQIPw4QOFI/cWqTty C9IpcN8K+gPXT9kAdJAM1GFFj4mCY5sSCAhxAISeERIYfsCQBKSIa0psJiCCcQlH/5/ipszL hOCUmQWatinh6C0ECCQSGjbMkKTiogBJIg5iwHPf0qBJfkoBoQnrEVoljwxAOUhAHiVRpX6o kEQB0aqZjPb86i8sWChWt/KVeKQAyQRvl8SVumEJQkgvjdhU2HURL799kYiFUi0kH3QEEJgl QjDS0yYa5DaUpQQppNFEtAqC/W8X0MxHNj+RLYJ2brdGiXz0VqbJBNUfIDxYUuCukcA3I/MT cVnz39zXm7heFFzKgEAE6jk5jjweE5KCpAuxObG2t9uL4A9y80YIWxFKUY24hUD9EvLIddAE A5nQZhck4uljmXyCMEgOMehVN8Uj+TzxQATINURTE81BQv/ZENsJglMRe2EW32AAEZOJhFF8 5JsSF2bYECJODLWIdI8NUiGJXpkIDIodCZGJQFKEwJWFGMo41zqZHDkQLtgtCKQIDoajYjmo 7NhEBUnKGAFnmewkRIj4WSeljw1OaaWQPXrnXxLvKPkBa09AJwIcOQpCpIK2TemgFCvqRx8X F3jgwKGIJnroBWi8mUqTT1pkZp9oUulnFPflN4VG5JxWEyQ7kSlmlJRO+mOlUtg4iKNEMJAA A8FxGo6nSNhZQJ4iOGlEiaamiWoUBL523iAGLFCGrMjQesSBEmXCKq+kntqrpVAwW2phYbKJ rC/KHoGrc72yyCNuu2bXBHoiJGj/hKqCHLntL90aQWaBQIpb2Ym/RoHWIgQUN4smcLzLLRMd bjJiuH9eKsW3BOg6wgDzGqQEgHLKRece6LbFoL18ShttQUOQpUloIfxhk4jdEUFxxVItp1+w mqhb7pnTVkkOEfv28+IQK7P8AQf6CdFZJpoirPC0/fzbT3uGdOmzVBYELQS7gmhpdL5I31zE ASd74jARMT4t1zRSj5DzIP5eXbOaWhcxgJ0rpl2EBQ3UbffdeOMdddkDlOx3yU2A8HcIO4c8 +J5FCH744owjLnQACORJAAAhpFz25ZhnrvnmnHfu+eeghy766KSXbvrpqKeu+uqst+666wsQ CQJlJxFh/8cCDKgXgB0A0MaAHYULEcCOu59S+xAA6IpHEb/XAVsAExkj3R+J8F4H8v4WsEAd CCi7gDrDx2aHeAEcPAICkpBFBCEhLGBAvzwZ4DetCDQsOAFMc3MZJR+qP0R0yBNIAuy3APyF 7E48GdXDnJGgkt3Cb//zVAEIgJgD3AI2ihBPCC7zCKcEQDzVAKEIyuA/IXmqGZHZ4FlGOAS7 FE6Fl0DIaUo4Auq4gnLcgJ8QyOIpA+jKAM9TyYcOaAQReAoACUghASIDAALkB4YjAIDVhiAA VzAgPzQ04hBcpL8jAGAiAhgiifYnRTFlkYf2UUYCrIY+bmBRBNJ5FVmCQ0MTiv8hXcIhwGik KL9LXEYASYQIHIXwKiJG0HbKCEECthAZPOLMfDwhIxezuB9e4NCOkRxCc8pQRSKQRQ999CS1 DjmCOkZRGZQrnx6geCsgegsrzTnFGYmAgEROrg7F0aIow0LGETBgiZTk4iVrqKwDVEeKC6RV JydRNFPq0pQrCWAUCwHFh90CAOpZgEK0uUtSprGLkvFd0TLpRyHIj5LMXIsyHAlO4W0jAJqq hlNAQAtaOXOGg0SkNO0iOBYpIAESssutYDPLO2pQXAgQIwDESM5J8IIstFjfEOQ3TJUUwaJb DE8tW3XLOhhAHfcUWm+EM9JvVrKajzICQM1XUBAAtKGosSmp2XSZm15OzRkSFQLXQjlT8RBI WXRA4ERh84jIhFQIzThF/ZhIjI+I5Ai/NAKBrJZFgAZCANPrqJb484dkLMWmZvBVDW3nVSqy TxG+IUt1QLDEIvjQkOvz1HcIELv3+WuYBLpMQkNwQWpUrhgC8dsB/KO4ktGmsHJL3GjoGZvA EqEATkmc3xI7CXv65gCjgeyughOAksFqFrQinOFKFq/Xmfa0RQgCADs= --------------E63235C68732C863C955B2C7-- --------------B5DB6867702107CCCBAC68E5--