lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: weird drastic query latency during performance testing and DIH import delay after performance testing
Date Fri, 18 Jul 2014 01:47:04 GMT
This is very strange. I have no idea why DIH is
taking so long. What happens if you execute the DIH
query in some SQL front end? It's possible that DIH
is just taking forever to execute the SQL query because
of how it's written.

I'm having trouble following the query results, but again
this is super-slow. How many docs do you have in
your index? How much memory have you allocated
for your JVM? Your query is actually pretty simple, so I have
no clue whatsoever why your response times are
so slow. But this is such bad performance that something
major is wrong.

I'd approach it one problem at a time. Understand what's
happening with your queries, _then_ tackle DIH (or the other
way around)...

Best,
Erick


On Wed, Jul 16, 2014 at 2:03 AM, YouPeng Yang <yypvsxf19870706@gmail.com>
wrote:

> Hi
>   I build my SolrCloud using Solr 4.6.0 (java version:1.7.0_45). In my
> cloud,I have a collection with 30 shard,and each shard has one replica.
> each core of the shard contains nearly  50 million docs  that is 15GB in
> size,so does the replica.
>   Before applying my cloud in the real world,I do a performance test with
> JMeter 2.11.
>   The scenario of the my test is simple:100 threads sending requests for 20
> seconds ,and these requests are only sent to  a specific core of a specific
> shard.the request is similar to the following :
>  http://IP:port/solr/tv_201407/select?q=*:*&fq=BEGINTIME:["2014-06-01
> 00:00:00"+TO+*]+AND+(CONTACT:${user})+AND (TV_STATE:"00")&shards=tv_201407
> &rows=2000&sort=BEGINTIME+desc.
>
>   I encountered the drastic  query latency during performance testing and
> DIH import delay after performance testing.Please help me. I have tested
>  several times and get the same problem and can not handle it by myself.Any
> suggestion will be apprecaited.
>
>  The following steps describes what I have done .
>
> Step 1: Before the test,the DIH import job is very fast.As the statistics
> [1], the DIH importing takes only 1s for 10 docs.
> [1]-----------------------------------
> Indexing completed. Added/Updated: 10 documents. Deleted 0 documents.
> (Duration: 01s)
> Requests: 1 (1/s), Fetched: 10 (10/s), Skipped: 0, Processed: 10 (10/s)
> Started: less than a minute ago
> ---------------------------------------
>
> Step 2:  Then ,Doing the test under the caches are cleaned. The summery
> statistics data is as [2]. Although I have clean the caches,I never think
> the query latency becomes so drastic that it cannot be acceptable in my
> real application.
>   The red font describes the latency of the query performance test on the
> core tv_201407 of the shard tv_201407 .
>
>   So would you experts can give some hints about the drastic  query latency
> ?
>
> [2]-----------------------------------
> [solr@solr2 test]$ ../bin/jmeter.sh  -n -t solrCoudKala20140401.jmx  -l
> logfile_solrCloud_20.jtl
> Creating summariser <aggregate>
> Created the tree successfully using solrCoudKala20140401.jmx
> Starting the test @ Wed Jul 16 15:59:28 CST 2014 (1405497568104)
> Waiting for possible shutdown message on port 4445
> aggregate +      1 in   8.1s =    0.1/s Avg:  8070 Min:  8070 Max:  8070
> Err:
>     0 (0.00%) Active: 100 Started: 100 Finished: 0
> aggregate +    103 in  13.4s =    7.7/s Avg:  8027 Min:  4191 Max:  8434
> Err:
>     0 (0.00%) Active: 97 Started: 100 Finished: 3
> aggregate =    104 in  13.4s =    7.7/s Avg:  8027 Min:  4191 Max:  8434
> Err:
>     0 (0.00%)
> aggregate +     96 in     7s =   14.5/s Avg:  6160 Min:  5295 Max:  6625
> Err:
>     0 (0.00%) Active: 0 Started: 100 Finished: 100
> aggregate =    200 in    15s =   13.6/s Avg:  7131 Min:  4191 Max:  8434
> Err:
>     0 (0.00%)
> Tidying up ...    @ Wed Jul 16 15:59:43 CST 2014 (1405497583461)
> ... end of run
> [solr@solr2 test]$
> ---------------------------------------
> Step 3:To be continued,after the test,I do the DIH importing job again
> using  the same import expresion.However the performance of the DIH becomes
> so unacceptable.
> to import  the 10 docs takes 2 m 15 s [3]!
>   Having noticing that ,solr can fetched the 10 docs fast,the processing is
> slow.
>
> [3]-----------------------------------
> *Indexing completed. Added/Updated: 10 documents. Deleted 0 documents.
> (Duration: 2m 15s)*
> Requests: 1 (0/s), Fetched: 10 (0/s), Skipped: 0, Processed: 10 (0/s)
> Started: about an hour ago
> ---------------------------------------
>
>  By the way. jvm gc goes normal,and there is no long full gc during the
> test. the load of my system(rhel 6.5) are also normal.
>
> Regards
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message