arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anson Qian (JIRA)" <>
Subject [jira] [Created] (ARROW-4736) [Go] Optimize memory usage for CSV writer
Date Fri, 01 Mar 2019 19:43:00 GMT
Anson Qian created ARROW-4736:

             Summary: [Go] Optimize memory usage for CSV writer
                 Key: ARROW-4736
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: Anson Qian

perhaps not for this PR, but, depending on the number of rows and cols this record contains,
this may be a very large allocation, and very big memory chunk.
it could be more interesting performance wise to write n rows instead of everything in one
big chunk.

also, to reduce the memory pressure on the GC, we should probably (re)use this slice-of-slice
of strings.


This message was sent by Atlassian JIRA

View raw message