lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-2679) Optimize memory usage in JavaBinCodec.readStr()
Date Thu, 08 Dec 2011 23:47:40 GMT


Yonik Seeley commented on SOLR-2679:

I think the right approach to increased efficiency here is to expose the byte[] of the FastInputStream.
It's not a super-simple approach since it needs to operate chunk-at-a-time and deal with multi-byte
chars crossing chunks.
The goal of eliminating the double-buffering seems fine though - I think we can leave this
issue open.
> Optimize memory usage in JavaBinCodec.readStr()
> -----------------------------------------------
>                 Key: SOLR-2679
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java, Response Writers, update
>    Affects Versions: 3.3
>            Reporter: Maxim Valyanskiy
>             Fix For: 3.6, 4.0
>         Attachments: SOLR-2679.patch
> In Solr 3.3, JavaBinCodec.readStr() reads complete string into byte[] and then converts
it into String. FastInputStream is already buffered, so this it not necessary and may take
a lot of memory if data string is large.
> This patch replaces usage of big byte[] with usage of dis.readUnsignedByte() for getting
characters directly from stream.
> Patch reduces memory usage and may slightly improve performance of reading text data
in javabin-codec.
> JavaBinCodec.readStr() function already has unit test in TestJavaBinCodec, no additional
unit test is required.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message