lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Solr Wiki] Update of "SolrPerformanceData" by domtheo
Date Tue, 05 Mar 2013 08:08:10 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "SolrPerformanceData" page has been changed by domtheo:
http://wiki.apache.org/solr/SolrPerformanceData?action=diff&rev1=24&rev2=25

  
  Pleae try to give as many specifics as you can regarding:
  
+  * [[http://vamostech.com/gps-tracking|GPS Tracking]]
+  * [[http://vamostech.com/gps-tracking|GPS Tracker]]
+  * [[http://www.pintuvariasi.com|Pintu Aluminium]]
+  * [[http://www.trimasjaya.com/pintu-dan-jendela/index.html|Pintu dan Jendela]]
   * The Hardware and OS you used
   * The version of Solr you used
   * The Servlet Container and JVM you used
@@ -16, +20 @@

  
  See also: SolrPerformanceFactors
  
- See also: [[http://lucene.apache.org/java/2_4_0/benchmarks.html|Lucene's benchmark page]]
and this page on hardware considerations [[http://wiki.statsbiblioteket.dk/summa/Hardware|from
Summa]] (which is also based on Lucene)
+ See also: and this page on hardware considerations(which is also based on Lucene)
  
  == JobHits job search engine ==
- JobHits is a Solr powered [[http://jobhits.co.uk/|search engine for jobs]] since the start
on May 2009. JobHits has 3 localized version for UK, [[http://jobhits.net/|US]] and [[http://jobhits.net/|Canada]].
Each website run on a 2.0 GHz Dual Quad Core dedicate server with 16GB RAM. The server is
used for continuously crawling new data, indexing and for the public job search site.
+ JobHits is a Solr powered since the start on May 2009. JobHits has 3 localized version for
UK. Each website run on a 2.0 GHz Dual Quad Core dedicate server with 16GB RAM. The server
is used for continuously crawling new data, indexing and for the public job search site.
  
  New jobs are being added to the index every 1 minute at the rate of more than 40,000 new
documents per day. On the highest loaded site JobHits.co.uk, there are about 2,1 million queries
to Solr daily at the rate of 24.235846 requests per second and the average query time is only
34 ms. Below are one sample statistic of the standard search handler component:
  
@@ -34, +38 @@

  avgRequestsPerSecond : 24.235846
  }}}
  == CNET Shopper.com ==
- The numbers below are from testing done by CNET prior to launching a Solr powered [[http://www.shopper.com|Shopper.com]]
search page.  Shopper.com uses a modified version of the DisMaxRequestHandler which also does
some faceted searching to pick categories for the page navigation options.  On a typical request,
the handler fetches the !DocSets for 1500-2000 queries and intersects each with the !DocSet
for the main search results.
+ The numbers below are from testing done by CNET prior to launching a Solr powered search
page.  Shopper.com uses a modified version of the DisMaxRequestHandler which also does some
faceted searching to pick categories for the page navigation options.  On a typical request,
the handler fetches the !DocSets for 1500-2000 queries and intersects each with the !DocSet
for the main search results.
  
  The plugin itself uses configuration nearly identical to the DisMaxRequestHandler.  To give
you an idea of the types of queries that it generates:
  
@@ -65, +69 @@

         75th percentile (secs):   0.027   0.040   0.072   0.108
         50th percentile (secs):   0.017   0.024   0.042   0.063
  }}}
- Mailing list post [[http://www.nabble.com/forum/ViewPost.jtp?post=4487784&framed=y|"Two
Solr Announcements: CNET Product Search and DisMax"]] describes a little more about Solr and
CNET.
+ Mailing list post  describes a little more about Solr and CNET.
  
  == Netflix ==
- Walter Underwood reports that [[http://www.netflix.com|Netflix]]'s site search switched
to being powered by Solr the week of 9/17/07:
+ Walter Underwood reports that 's site search switched to being powered by Solr the week
of 9/17/07:
  
   . Here at Netflix, we switched over our site search to Solr two weeks ago. We've seen zero
problems with the server. We average 1.2 million queries/day on a 250K item index. We're running
four Solr servers with simple round-robin HTTP load-sharing. This is all on 1.1. I've been
too busy tuning to upgrade.
  
- (See http://www.nabble.com/forum/ViewPost.jtp?post=13009485&framed=y)
  
  Walter also reported some figures from their testing phase:
  
@@ -80, +83 @@

  
  At least for these test figures, they were not using fuzzy search, facets, or highlighting.
  
- (See http://www.nabble.com/forum/ViewPost.jtp?post=12906462&framed=y)
+ 
  
  == Discogs.com ==
- Solr powers keyword search on [[http://www.discogs.com/|Discogs.com]]. From the [[http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200611.mbox/<3f732c0b0611060921q1c67185fkb454a901a6abb998@mail.gmail.com>|email
archive]] ([[http://www.nabble.com/forum/ViewPost.jtp?post=7203032&framed=y|alternate
copy on nabble]])...
+ Solr powers keyword search on. From the...
  
  {{{
  I've been using Solr for keyword search on Discogs.com for a few
@@ -96, +99 @@

  the time.
  }}}
  == HathiTrust Large Scale Solr Benchmarking ==
- [[http://www.hathitrust.org|HathiTrust]] ''makes the digitized collections of some of the
nation’s great research libraries available for all.''  We currently have slightly over
5 million full-text books indexed.  Our production index is spread across 10 shards on 4 machines.
With a total index size of over 2 Terabytes, our biggest bottleneck is disk I/O.  We did reduce
that significantly using CommonGrams, but disk I/O is still the bottleneck for performance.
+ 'makes the digitized collections of some of the nation’s great research libraries available
for all.''  We currently have slightly over 5 million full-text books indexed.  Our production
index is spread across 10 shards on 4 machines. With a total index size of over 2 Terabytes,
our biggest bottleneck is disk I/O.  We did reduce that significantly using CommonGrams, but
disk I/O is still the bottleneck for performance.
  
- On our production index, the average Solr response time is around 200 ms, median response
time 90 ms, 90th percentile about 450 ms, and 99th percentile about 1.4 seconds.  Details
on the hardware are available at [[http://www.hathitrust.org/blogs/large-scale-search/new-hardware-searching-5-million-volumes-full-text|New
hardware for searching 5 million plus volumes]]  Some details on performance are available
at: [[http://www.hathitrust.org/blogs/large-scale-search/performance-5-million-volumes|Performance
at 5 million volumes]].  Background and updates available at:[[http://www.hathitrust.org/blogs/large-scale-search|The
HathiTrust Large Scale Search blog]]
+ On our production index, the average Solr response time is around 200 ms, median response
time 90 ms, 90th percentile about 450 ms, and 99th percentile about 1.4 seconds.  Details
on the hardware are available at  Some details on performance are available at:.  Background
and updates available at
  
- == Zvents ==
- [[http://www.zvents.com|Zvents]] serves more than 8 millions users monthly with engaging
local content.  We've used Solr for several years and have achieved very high performance
and reliability.  User queries are served by a cluster of 8 machines, each having 16Gigs of
memory and 4 cores.  Our search index contains over 4 million documents.  An average week
day sees a maximum 80qps with an average latency of 40ms.  Leading up to New Years, we'll
see ten times this level.  To support huge fluctuations in our capacity needs, we run a nightly
load test against a single production class machine.  The load test itself uses JMeter, a
copy of production access logs, and a copy of the production index.  The load testing machine
is subjected to 130qps and delivers an average latency of 150ms.
+ == Zvents == serves more than 8 millions users monthly with engaging local content.  We've
used Solr for several years and have achieved very high performance and reliability.  User
queries are served by a cluster of 8 machines, each having 16Gigs of memory and 4 cores. 
Our search index contains over 4 million documents.  An average week day sees a maximum 80qps
with an average latency of 40ms.  Leading up to New Years, we'll see ten times this level.
 To support huge fluctuations in our capacity needs, we run a nightly load test against a
single production class machine.  The load test itself uses JMeter, a copy of production access
logs, and a copy of the production index.  The load testing machine is subjected to 130qps
and delivers an average latency of 150ms.
  

Mime
View raw message