commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ortwin Glück (JIRA) <j...@apache.org>
Subject [jira] Updated: (SANDBOX-166) Improve memory use
Date Wed, 09 Aug 2006 12:45:14 GMT
     [ http://issues.apache.org/jira/browse/SANDBOX-166?page=all ]

Ortwin Glück updated SANDBOX-166:
---------------------------------

    Attachment: optimized.png

These are the figures parsing the same file as before but with the patch applied.
The file contained 47632 rows of 10 columns each. The total file size is around 4MB.

You can clearly see:
- char[] are now down to the number of fields in the file: 476.741 now vs. 1.620.351 befor
- A single CSVParser$Token is now reused: 1 now vs. 476.315 before
- The intermediate String/StringBuffer objects are not reported, but they basically are the
same number as char[] and now 4 CharBuffer instances are used only.



> Improve memory use
> ------------------
>
>                 Key: SANDBOX-166
>                 URL: http://issues.apache.org/jira/browse/SANDBOX-166
>             Project: Commons Sandbox
>          Issue Type: Improvement
>          Components: CSV
>    Affects Versions: Nightly Builds
>            Reporter: Ortwin Glück
>         Attachments: optimized.png, patch.txt, profile.png
>
>
> The parser is currently a real memory burner. I fed it a 4MB CSV file and ran the TPTP
profiler.  As you can see the parser creates around 100MB of garbage whereas it could (in
really optimized) use around 4MBs.  Such figures are not acceptable within a server environment.
Please attach insights and patches to this issue report.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message