lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <>
Subject [jira] [Commented] (SOLR-9636) Add support for javabin for /stream, /sql internode communication
Date Tue, 27 Dec 2016 23:15:58 GMT


Joel Bernstein commented on SOLR-9636:

I finally had the time to test out the javabin writer with the /export handler and streaming
stack. My initial findings are really good. Here is a summary:

1) Currently testing must be done on branch_6x. There is a bug in master which breaks the
/export handler. I haven't gotten to the bottom yet but I'm pretty sure it was introduced
with the new docValues iterator API which is only in master. I will open a ticket for this
bug shortly and see if I can fix the problem.

But testing in branch_6x is better anyway as it won't be testing both the docValues iterator
API performance at the same time as the javabin /export performance.

2) For my test I worked on a single Solr instance with a single data shard loaded with 10,000,000
small documents. I also created a worker collection with 5 shards. The I ran the following
expression with and without the javabin writer.
parallel(collection2, workers=5, sort="test_s desc", 
            rollup(over="test_s", sum(price_f),
                      search(collection1, q=*:*, fl="test_s, price_f", sort="test_s desc",
qt="/export", wt="javabin", partitionKeys=test_s)))

> Add support for javabin for /stream, /sql internode communication
> -----------------------------------------------------------------
>                 Key: SOLR-9636
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Noble Paul
>            Assignee: Noble Paul
>             Fix For: master (7.0), 6.4
>         Attachments: SOLR-9636.patch
> currently it uses json, which is verbose and slow

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message