carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacky Li (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CARBONDATA-318) Implement an ExternalSorter that makes maximum usage of memory while sorting
Date Sat, 15 Oct 2016 05:37:20 GMT
Jacky Li created CARBONDATA-318:
-----------------------------------

             Summary: Implement an ExternalSorter that makes maximum usage of memory while
sorting
                 Key: CARBONDATA-318
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-318
             Project: CarbonData
          Issue Type: Sub-task
            Reporter: Jacky Li


External Sorter should sort in memory until it reach configured size, then spill to disk.
It should provide following interface:
1. insertRow/insertRowBatch: insert rows into the sorter
2. getIterator: return an iterator that iterate on sorted rows

External Sorter depends on FileWriterFactory to get a FileWriter to spill data into files.
FileWriterFactory should be provided by user. Multiple implementations are possible, like
writing into one folder or multiple folder



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message