carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xuchuanyin (JIRA)" <>
Subject [jira] [Created] (CARBONDATA-1837) Reusing old row to reduce memory consumption
Date Wed, 29 Nov 2017 13:48:00 GMT
xuchuanyin created CARBONDATA-1837:

             Summary: Reusing old row to reduce memory consumption
                 Key: CARBONDATA-1837
             Project: CarbonData
          Issue Type: Improvement
          Components: data-load
            Reporter: xuchuanyin
            Assignee: xuchuanyin
            Priority: Minor

In data converting process of data loading, Carbondata will convert each row to another row
by batch. 

Currently, it will create a new batch to store the converted rows, which I think can be optimized
to reuse the old row batch's space, thus will reduce memory consumption and GC overhead.

This message was sent by Atlassian JIRA

View raw message