lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2679) Optimize memory usage in JavaBinCodec.readStr()
Date Thu, 08 Dec 2011 21:21:40 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165575#comment-13165575
] 

Erick Erickson commented on SOLR-2679:
--------------------------------------

Does anyone have an opinion on this? It seems to me that the tradeoff is that the current
form of this code reads the bytes from a local array, where the proposed patch makes a method
call for every byte that's read, which seems like it'd be slower since the dis.readFully method
does an arrayCopy under the covers.

I'll volunteer to apply or close this, it seems like a time/space tradeoff. And the underlying
byte[] grows to a max size and gets re-used.

Since the class that's used to read data is called FastInputStream, my straw-man proposal
is that we do NOT apply this patch and I'll close the JIRA.
                
> Optimize memory usage in JavaBinCodec.readStr()
> -----------------------------------------------
>
>                 Key: SOLR-2679
>                 URL: https://issues.apache.org/jira/browse/SOLR-2679
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java, Response Writers, update
>    Affects Versions: 3.3
>            Reporter: Maxim Valyanskiy
>             Fix For: 3.6, 4.0
>
>         Attachments: SOLR-2679.patch
>
>
> In Solr 3.3, JavaBinCodec.readStr() reads complete string into byte[] and then converts
it into String. FastInputStream is already buffered, so this it not necessary and may take
a lot of memory if data string is large.
> This patch replaces usage of big byte[] with usage of dis.readUnsignedByte() for getting
characters directly from stream.
> Patch reduces memory usage and may slightly improve performance of reading text data
in javabin-codec.
> JavaBinCodec.readStr() function already has unit test in TestJavaBinCodec, no additional
unit test is required.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message