arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anson Qian (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ARROW-4736) [Go] Optimize memory usage for CSV writer
Date Fri, 01 Mar 2019 19:43:00 GMT
Anson Qian created ARROW-4736:
---------------------------------

             Summary: [Go] Optimize memory usage for CSV writer
                 Key: ARROW-4736
                 URL: https://issues.apache.org/jira/browse/ARROW-4736
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: Anson Qian


perhaps not for this PR, but, depending on the number of rows and cols this record contains,
this may be a very large allocation, and very big memory chunk.
it could be more interesting performance wise to write n rows instead of everything in one
big chunk.

also, to reduce the memory pressure on the GC, we should probably (re)use this slice-of-slice
of strings.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message