lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Kan <solrexp...@gmail.com>
Subject Re: Measuring SOLR performance
Date Tue, 06 Aug 2013 12:17:09 GMT
Hi Roman,

With fresh checkout, the reported admin_endpoint is:
http://localhost:8983/solr/admin. This url redirects to
http://localhost:8983/solr/#/ . I'm using solr 4.3.1. Is your tool
supporting this version?

Of three URLs you asked for, only the 3rd one gave response:

{"responseHeader":{"status":0,"QTime":5},"defaultCoreName":null,"initFailures":{},"status":{"metadata":{"name":"metadata","isDefaultCore":false,"instanceDir":"multicore/metadata/","dataDir":"/home/dmitry/projects/SOLR/solr4/solr-4.3.1/example/multicore/metadata/data/","config":"solrconfig.xml","schema":"schema.xml","startTime":"2013-08-06T12:10:43.438Z","uptime":260200,"index":{"numDocs":0,"maxDoc":0,"deletedDocs":0,"version":1,"segmentCount":0,"current":true,"hasDeletions":false,"directory":"org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/home/dmitry/projects/SOLR/solr4/solr-4.3.1/example/multicore/metadata/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d23b007;
maxCacheMB=48.0
maxMergeSizeMB=4.0)","userData":{},"sizeInBytes":65,"size":"65
bytes"}},"statements":{"name":"statements","isDefaultCore":false,"instanceDir":"multicore/statements/","dataDir":"/home/dmitry/projects/SOLR/solr4/solr-4.3.1/example/multicore/statements/data/","config":"solrconfig.xml","schema":"schema.xml","startTime":"2013-08-06T12:10:43.468Z","uptime":260172,"index":{"numDocs":2588869,"maxDoc":2588869,"deletedDocs":0,"version":45,"segmentCount":1,"current":true,"hasDeletions":false,"directory":"org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/home/dmitry/projects/SOLR/solr4/solr-4.3.1/example/multicore/statements/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f9b2b8;
maxCacheMB=48.0
maxMergeSizeMB=4.0)","userData":{"commitTimeMSec":"1375447171849"},"lastModified":"2013-08-02T12:39:31.849Z","sizeInBytes":597238578,"size":"569.57
MB"}}}}


The rest report 404.

Dmitry


On Mon, Aug 5, 2013 at 8:38 PM, Roman Chyla <roman.chyla@gmail.com> wrote:

> Hi Dmitry,
> So I think the admin pages are different on your version of solr, what do
> you see when you request... ?
>
> http://localhost:8983/solr/admin/system?wt=json
> http://localhost:8983/solr/admin/mbeans?wt=json
> http://localhost:8983/solr/admin/cores?wt=json
>
> If your core -t was '/solr/statements', the script should assume admin is
> at: /solr/admin (the script checks for /admin/system/cores - so that url
> already exists), thus I am guessing /admin/system is not there.
>
> if you can, please check out the latest version - the script will print its
> environment, the 'admin_endpoint' is the one that we are interested in.
> I'll update the docs, btw. you may want to use '-e statements' to indicate
> what core_name you want to harvest details for
>
> thanks,
>
> roman
>
>
>
>
>
>
>
> On Mon, Aug 5, 2013 at 6:22 AM, Dmitry Kan <solrexpert@gmail.com> wrote:
>
> > Hi Roman,
> >
> > No problem. Still trying to launch the thing..
> >
> > The query with the added -t parameter generated an error:
> >
> > 1. python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
> > ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60
> -R
> > test -t /solr/statements   [passed relative path to -q param]
> >
> > (as you can see I added -t param and made -q param simpler)
> >
> > Traceback (most recent call last):
> >   File "solrjmeter.py", line 1425, in <module>
> >     main(sys.argv)
> >   File "solrjmeter.py", line 1379, in main
> >     before_test = harvest_details_about_montysolr(options)
> >   File "solrjmeter.py", line 505, in harvest_details_about_montysolr
> >     system_data = req('%s/system' % options.admin_endpoint)
> >   File "solrjmeter.py", line 113, in req
> >     raise r
> > simplejson.decoder.JSONDecodeError: No JSON object could be decoded:
> line 1
> > column 0 (char 0)
> >
> >
> > The README.md on the github is somehow outdated, it suggests using -q
> > ./demo/queries/demo.queries, but there is no such path in the fresh
> > checkout.
> >
> > Nice to have the -t param.
> >
> > Dmitry
> >
> >
> > On Sat, Aug 3, 2013 at 5:01 AM, Roman Chyla <roman.chyla@gmail.com>
> wrote:
> >
> > > Hi Dmitry,
> > >
> > > Thanks, It was a toothing problem, fixed now, please try the fresh
> > checkout
> > > AND add the following to your arguments: -t /solr/core1
> > >
> > > that sets the path under which solr should be contacted, the handler is
> > set
> > > in the jmeter configuration, so if you were using different query
> > handlers
> > > than /select, it should be edited there (SolrQueryTest.jmx)
> > >
> > > I hope it works this time, the script is trying to guess the admin page
> > > (when one cannot be contacted - but if the new solr introduces some new
> > > paths, i may be wrong - i am short on time to investigate deeper)
> > >
> > >
> > > roman
> > >
> > >
> > > On Fri, Aug 2, 2013 at 7:27 AM, Dmitry Kan <solrexpert@gmail.com>
> wrote:
> > >
> > > > Hi Roman,
> > > >
> > > > Sure:
> > > >
> > > > python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
> > > > /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries -s
> > > localhost
> > > > -p 8983 -a --durationInSecs 60 -R test
> > > >
> > > > This is vanilla install (git clone) except for one change that I had
> to
> > > do
> > > > related to solr cores:
> > > >
> > > > > git diff
> > > > diff --git a/solrjmeter.py b/solrjmeter.py
> > > > index d18145a..7a0d2af 100644
> > > > --- a/solrjmeter.py
> > > > +++ b/solrjmeter.py
> > > > @@ -129,7 +129,7 @@ def check_options(options, args):
> > > >      if not options.serverName and not options.serverPort:
> > > >          error("You must specify both server and port")
> > > >
> > > > -    options.query_endpoint = 'http://%s:%s/solr' %
> > (options.serverName,
> > > > options.serverPort)
> > > > +    options.query_endpoint = 'http://%s:%s/solr/core1' %
> > > > (options.serverName, options.serverPort)
> > > >
> > > >      jmx_options = []
> > > >      for k, v in options.__dict__.items():
> > > >
> > > >
> > > >
> > > > Dmitry
> > > >
> > > >
> > > > On Thu, Aug 1, 2013 at 6:41 PM, Roman Chyla <roman.chyla@gmail.com>
> > > wrote:
> > > >
> > > > > Dmitry,
> > > > > Can you post the entire invocation line?
> > > > > roman
> > > > >
> > > > >
> > > > > On Thu, Aug 1, 2013 at 7:46 AM, Dmitry Kan <solrexpert@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi Roman,
> > > > > >
> > > > > > When I try to run with -q
> > > > > > /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries
> > > > > >
> > > > > > here what is reported:
> > > > > > Traceback (most recent call last):
> > > > > >   File "solrjmeter.py", line 1390, in <module>
> > > > > >     main(sys.argv)
> > > > > >   File "solrjmeter.py", line 1309, in main
> > > > > >     tests = find_tests(options)
> > > > > >   File "solrjmeter.py", line 461, in find_tests
> > > > > >     with changed_dir(pattern):
> > > > > >   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
> > > > > >     return self.gen.next()
> > > > > >   File "solrjmeter.py", line 229, in changed_dir
> > > > > >     os.chdir(new)
> > > > > > OSError: [Errno 20] Not a directory:
> > > > > > '/home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries'
> > > > > >
> > > > > > Best,
> > > > > >
> > > > > > Dmitry
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Jul 31, 2013 at 7:21 PM, Roman Chyla <
> > roman.chyla@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Dmitry,
> > > > > > > probably mistake in the readme, try calling it with -q
> > > > > > > /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries
> > > > > > >
> > > > > > > as for the base_url, i was testing it on solr4.0, where
it
> tries
> > > > > > contactin
> > > > > > > /solr/admin/system - is it different for 4.3? I guess I
should
> > make
> > > > it
> > > > > > > configurable (it already is, the endpoint is set at the
> > > > > check_options())
> > > > > > >
> > > > > > > thanks
> > > > > > >
> > > > > > > roman
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Jul 31, 2013 at 10:01 AM, Dmitry Kan <
> > solrexpert@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Ok, got the error fixed by modifying the base solr
ulr in
> > > > > solrjmeter.py
> > > > > > > > (added core name after /solr part).
> > > > > > > > Next error is:
> > > > > > > >
> > > > > > > > WARNING: no test name(s) supplied nor found in:
> > > > > > > >
> > > ['/home/dmitry/projects/lab/solrjmeter/demo/queries/demo.queries']
> > > > > > > >
> > > > > > > > It is a 'slow start with new tool' symptom I guess..
:)
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Jul 31, 2013 at 4:39 PM, Dmitry Kan <
> > > solrexpert@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Roman,
> > > > > > > >>
> > > > > > > >> What  version and config of SOLR does the tool
expect?
> > > > > > > >>
> > > > > > > >> Tried to run, but got:
> > > > > > > >>
> > > > > > > >> **ERROR**
> > > > > > > >>   File "solrjmeter.py", line 1390, in <module>
> > > > > > > >>     main(sys.argv)
> > > > > > > >>   File "solrjmeter.py", line 1296, in main
> > > > > > > >>     check_prerequisities(options)
> > > > > > > >>   File "solrjmeter.py", line 351, in check_prerequisities
> > > > > > > >>     error('Cannot contact: %s' % options.query_endpoint)
> > > > > > > >>   File "solrjmeter.py", line 66, in error
> > > > > > > >>     traceback.print_stack()
> > > > > > > >> Cannot contact: http://localhost:8983/solr
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> complains about URL, clicking which leads properly
to the
> > admin
> > > > > > page...
> > > > > > > >> solr 4.3.1, 2 cores shard
> > > > > > > >>
> > > > > > > >> Dmitry
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Jul 31, 2013 at 3:59 AM, Roman Chyla <
> > > > roman.chyla@gmail.com
> > > > > > > >wrote:
> > > > > > > >>
> > > > > > > >>> Hello,
> > > > > > > >>>
> > > > > > > >>> I have been wanting some tools for measuring
performance of
> > > SOLR,
> > > > > > > similar
> > > > > > > >>> to Mike McCandles' lucene benchmark.
> > > > > > > >>>
> > > > > > > >>> so yet another monitor was born, is described
here:
> > > > > > > >>>
> > > > > > >
> > > > >
> > >
> http://29min.wordpress.com/2013/07/31/measuring-solr-query-performance/
> > > > > > > >>>
> > > > > > > >>> I tested it on the problem of garbage collectors
(see the
> > blogs
> > > > for
> > > > > > > >>> details) and so far I can't conclude whether
highly
> > customized
> > > G1
> > > > > is
> > > > > > > >>> better
> > > > > > > >>> than highly customized CMS, but I think interesting
details
> > can
> > > > be
> > > > > > seen
> > > > > > > >>> there.
> > > > > > > >>>
> > > > > > > >>> Hope this helps someone, and of course, feel
free to
> improve
> > > the
> > > > > tool
> > > > > > > and
> > > > > > > >>> share!
> > > > > > > >>>
> > > > > > > >>> roman
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message