From solr-user-return-143207-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Thu Aug 23 11:57:59 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 0810D18061A for ; Thu, 23 Aug 2018 11:57:58 +0200 (CEST) Received: (qmail 98618 invoked by uid 500); 23 Aug 2018 09:57:57 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 98606 invoked by uid 99); 23 Aug 2018 09:57:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Aug 2018 09:57:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id EEB191A20FE for ; Thu, 23 Aug 2018 09:57:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.64 X-Spam-Level: ** X-Spam-Status: No, score=2.64 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_INFOUSMEBIZ=0.75, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id L19HFuHRLnsm for ; Thu, 23 Aug 2018 09:57:54 +0000 (UTC) Received: from mail-pl0-f52.google.com (mail-pl0-f52.google.com [209.85.160.52]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id BBFE05F11D for ; Thu, 23 Aug 2018 09:57:53 +0000 (UTC) Received: by mail-pl0-f52.google.com with SMTP id s17-v6so2184290plp.7 for ; Thu, 23 Aug 2018 02:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=xrVwpf1LdE2bXBG7FmarR3J4jl8SrZJzv6oZ9egPPwE=; b=gLqm5Rcj5fi24IS637hMYzLM2veJM/1qV67em1CZgDOw5Y0UtAL8mHucRjJL3OAhx8 kb9L9bz4Yzg9NNCutPzs/KT7Ix8CDvxNpgdzeDPMOIFYD+jW4rbtwGctCL0wiVCfJcxt tM1hOYf+bwFoRYWZM0XZbI1QIDoI2YkV2/M/J5MDCMnrXJmzd4HXZzZcaTFzzAV/QLEO AH3xaXVAhwOmWo/PiDa879RHqzij1BgBbLAsDUWFQZJIN5yw7j5mLsJnufwkXz3q/eSD RrhApDk9WrAZhJ+Emt4R9LOrdjx4l9l422fcx1aQ/4CG28ZR1aleGyr655JXlY3mVM/S PJnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=xrVwpf1LdE2bXBG7FmarR3J4jl8SrZJzv6oZ9egPPwE=; b=r0mT0OdUdSmTI1g70qyR52l924McfLTjY052J5LOdtS4TYNmY0QdwsaEW8lOOtzaMm cKElrHq1qB6YUWu+AWMq7mdYDRZR12aDWjixYniVLJo6OGKm9wPk2K4n2If3IkJh+I/J HHDyTtQWX4H/VYr3zzoG3kTJaTsNgMVGWtJ39ffwtxbI8VN118Awkc9KzRFUAHB/wghF /3C3Arexke8wmUYylpyIOiLX3vu5ozjUhXIWO0ozP1Wc8lHjvSgZj4x7A/HohrMit5o7 sHr1I9D5Uoy3TXPBiPs3OEWbR7nxpx4vrwA0xigtAYR8GpivFwoPZTkhTvlo0smv+zv3 GljA== X-Gm-Message-State: APzg51BnfcSDIYTvdcC2+c/wkfvQkxHgSOjhFKDm1V2ufEQjoQNdkUL6 ck/rmKv+6Bhb147rLQAp79Ddnu4sZ/iZsvrqBWFbk+Hd X-Google-Smtp-Source: ANB0VdavWsQo/QqKHYlMAHpDs2lCQXKOBuJ0sOAgx59nNPPiXNddgpvY5Ur4l/g7N5Y1H1LKYr7mgSeiG/DKpa76WMA= X-Received: by 2002:a17:902:b68d:: with SMTP id c13-v6mr9319586pls.167.1535018272719; Thu, 23 Aug 2018 02:57:52 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: zhenyuan wei Date: Thu, 23 Aug 2018 17:57:41 +0800 Message-ID: Subject: =?UTF-8?Q?Re=3A_How_to_trace_one_query=EF=BC=9Fthe_debug=2FdebugQuery_in?= =?UTF-8?Q?fo_are_not_enough_to_find_out_why_a_query_is_slow?= To: solr-user@lucene.apache.org Content-Type: multipart/alternative; boundary="000000000000de87a60574174a0d" --000000000000de87a60574174a0d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I have 4 solr server, each allocated 6GB=E3=80=82My dataset on HDFS is 787G= B, 2 billion documents totally=EF=BC=8Ceach document is 300 Bytes=E3=80=82 Follow is my= cache related configuration=E3=80=82 20 200 zhenyuan wei =E4=BA=8E2018=E5=B9=B48=E6=9C=8823=E6=97= =A5=E5=91=A8=E5=9B=9B =E4=B8=8B=E5=8D=885:41=E5=86=99=E9=81=93=EF=BC=9A > Thank you very much to answer. @Jan H=C3=B8ydahl > My query is simple, just wildcard last 2 char in this query=EF=BC=88have = more > other query to optimize=EF=BC=89 > > curl " > http://emr-worker-1:8983/solr/collection005/query?q=3Dv10_s:OOOOOOOOVVVVV= VVVYY*&rows=3D10&&fl=3Did&echoParams=3Dall > " > { > "responseHeader":{ > "zkConnected":true, > "status":0, > "*QTime":125,* > "params":{ > "q":"v10_s:OOOOOOOOVVVVVVVVYY*", > "df":"_text_", > "echoParams":"all", > "indent":"true", > "fl":"id", > "rows":"10", > "wt":"json"}}, > "response":{"numFound":118181,"start":0,"maxScore":1.0,"docs":[ > ....... > }} > > The first time query , it return slow, second time it returns very fast= =E3=80=82It > maybe cached already=E3=80=82 > With "debugQuery=3Dtrue&shards.info=3Dtrue", I can see all shards toke = 40 > or more millisecond =E3=80=82 > The second time the query executes =EF=BC=8C all shards query spend only = 1~5 > millisecond=E3=80=82 > But I dont know how to optimize the first time the query > execute=EF=BC=8CfilterCache?QueryResultCache? > or hdfs blockCache? and how much RAM I should allocate to them=EF=BC=9F > > I want to turn down QTime to 50ms or less in the first time query execute= s=E3=80=82 > > > > > Jan H=C3=B8ydahl =E4=BA=8E2018=E5=B9=B48=E6=9C=88= 23=E6=97=A5=E5=91=A8=E5=9B=9B =E4=B8=8B=E5=8D=885:19=E5=86=99=E9=81=93=EF= =BC=9A > >> Hi, >> >> With debugQuery you see the timings. What component spends the most time= ? >> With shards.info=3Dtrue you see what shard is the slowest, if your index >> is sharded. >> With echoParams=3Dall you get the full list of query parameters in use, >> perhaps you spot something? >> If you start Solr with -v option then you get more verbose logging in >> solr.log which may help >> >> Can you share with us how your query looks like, including all parameter= s >> from the section with echoParams=3Dall enabled? >> >> -- >> Jan H=C3=B8ydahl, search solution architect >> Cominvent AS - www.cominvent.com >> >> > 23. aug. 2018 kl. 11:09 skrev zhenyuan wei : >> > >> > Hi all=EF=BC=8C >> > I do care query performance, but do not know how to find out the >> reason >> > why a query so slow. >> > *How to trace one query=EF=BC=9F*the debug/debugQuery info are not eno= ugh to >> find >> > out why a query is slow=E3=80=82 >> > >> > >> > Thanks a lot~ >> >> --000000000000de87a60574174a0d--