lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Sun (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-10317) Solr Nightly Benchmarks
Date Wed, 07 Jun 2017 01:56:18 GMT

    [ https://issues.apache.org/jira/browse/SOLR-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039986#comment-16039986
] 

Michael Sun edited comment on SOLR-10317 at 6/7/17 1:55 AM:
------------------------------------------------------------

bq.  any thoughts, please?

Well, this is a good topic. Framework is fun and always a tradeoff. :) On one side, we should
not over invest on framework because the tests and numbers produced by tests are things really
matter. That's the target we should focus on. On the other hand, if framework is under invested,
in long run there is high cost. One way to understand the cost is to look how many Solr performance
frameworks there are already, not to mention a few not published (but I personally know).

I am one of guys who built my own (going to be open source soon, adding one more into the
collection). One of the motivation for me to build one is: developers only need to extend
framework but not rebuild one for all Solr performance work in near future. In my option this
is important and should be part of the goals of this project. And be part of the discussion
about framework choice too (no matter which one is chosen). 




was (Author: michael.sun):
bq.  any thoughts, please?

Well, this is a good topic. Framework is fun and always a tradeoff. :) On one side, we should
not over invest on framework because the tests and numbers produced by tests are things really
matter. That's the target we should focus on. On the other hand, if framework is under invested,
in long run there is high cost. One way to understand the cost is to look how many Solr performance
frameworks there are already, not to mention a few not published (but I personally know).

I am one of guys who built my own (going to be open source soon, adding one more into the
collection). One of the motivation for me to build one is 'only need to extend framework but
not rebuild one for all Solr performance work in near future'. In my option this is important
and should be part of the goals of this project. And be part of the discussion about framework
choice too (no matter which one is chosen).




> Solr Nightly Benchmarks
> -----------------------
>
>                 Key: SOLR-10317
>                 URL: https://issues.apache.org/jira/browse/SOLR-10317
>             Project: Solr
>          Issue Type: Task
>            Reporter: Ishan Chattopadhyaya
>              Labels: gsoc2017, mentor
>         Attachments: changes-lucene-20160907.json, changes-solr-20160907.json, managed-schema,
Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx, Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf,
solrconfig.xml
>
>
> Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be found here,
https://home.apache.org/~mikemccand/lucenebench/.
> Preferably, we need:
> # A suite of benchmarks that build Solr from a commit point, start Solr nodes, both in
SolrCloud and standalone mode, and record timing information of various operations like indexing,
querying, faceting, grouping, replication etc.
> # It should be possible to run them either as an independent suite or as a Jenkins job,
and we should be able to report timings as graphs (Jenkins has some charting plugins).
> # The code should eventually be integrated in the Solr codebase, so that it never goes
out of date.
> There is some prior work / discussion:
> # https://github.com/shalinmangar/solr-perf-tools (Shalin)
> # https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md (Ishan/Vivek)
> # SOLR-2646 & SOLR-9863 (Mark Miller)
> # https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
> # https://github.com/lucidworks/solr-scale-tk (Tim Potter)
> There is support for building, starting, indexing/querying and stopping Solr in some
of these frameworks above. However, the benchmarks run are very limited. Any of these can
be a starting point, or a new framework can as well be used. The motivation is to be able
to cover every functionality of Solr with a corresponding benchmark that is run every night.
> Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure [~shalinmangar]
and [~markrmiller@gmail.com] would help here.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message