lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <>
Subject [jira] Updated: (SOLR-486) Support binary formats for QueryresponseWriter
Date Tue, 15 Apr 2008 01:59:04 GMT


Yonik Seeley updated SOLR-486:

    Attachment: SOLR-486.patch

OK, latest version attached.

The bad:
 - more code, a more complex codec, harder to debug with a hex editor

The good:
- 2/3 the size of the previous binary codec (50% improvement and 1/3 the size of XML)
   - certain common types like named list, array, and string pack a size right in the tag
   - variable length int and long values.. saves lots of space for facet counts, etc (first
     bits of vint are packed in the tag byte)

- performance results: (9 times faster than xml, encoding + decoding)
  writer xml, size=6885, encodeRate=627, decodeRate=883
  writer json, size=3556, encodeRate=3084, decodeRate=N/A
  writer javabin-v1, size=3170
  writer javabin-current, size=2153, encodeRate=6095, decodeRate=8695 

- performance changes:
  - FastInputStream, FastOutputStream: unsynchronized buffered streams
  - toObject() changes to sortable numerics - bypass string and go straight to numeric type
  - tons of other little things, including everything that was used to decrease the size

- string values can now be > 64K (Java writeUTF is limited to that size)

- memory use changes: only "explode" one doc at a time in a doclist

- testing changes: more thorough random testing... (good thing too - found a long encoding

Noble: I started from your second-to-last patch (we overlapped in the Document fixes...) I
diffed your patches and I believe I saw everything.

TODO: preserve SimpleOrderedMap vs NamedList?
TODO: make SolrJ default?

> Support binary formats for QueryresponseWriter
> ----------------------------------------------
>                 Key: SOLR-486
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java, search
>            Reporter: Noble Paul
>            Assignee: Yonik Seeley
>            Priority: Minor
>             Fix For: 1.3
>         Attachments: SOLR-486.patch, SOLR-486.patch, SOLR-486.patch, SOLR-486.patch,
SOLR-486.patch, SOLR-486.patch, SOLR-486.patch
> QueryResponse writer only allows text data to be written.
> So it is not possible to implement a binary protocol . Create another interface which
has a method 
> write(OutputStream os, SolrQueryRequest request, SolrQueryResponse response)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message