carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacky Li (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-318) Implement an ExternalSorter that makes maximum usage of memory while sorting
Date Sun, 16 Oct 2016 01:21:20 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jacky Li updated CARBONDATA-318:
--------------------------------
    Description: 
Change SortDataRows into an External Sorter, it should sort in memory until it reach configured
size, then spill to disk. It should provide following interface:
1. addRow:  insert rows into the sorter.

2. getIterator: will return an iterator that iterates on sorted rows, the sorted row could
come from memory or files


  was:
External Sorter should sort in memory until it reach configured size, then spill to disk.
It should provide following interface:
1. insertRow/insertRowBatch: will take an Iterator as input and insert rows from the iterator
into the sorter.

2. getIterator: will return an iterator that iterates on sorted rows, the sorted row could
come from memory or files



> Implement an ExternalSorter that makes maximum usage of memory while sorting
> ----------------------------------------------------------------------------
>
>                 Key: CARBONDATA-318
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-318
>             Project: CarbonData
>          Issue Type: Sub-task
>            Reporter: Jacky Li
>             Fix For: 0.2.0-incubating
>
>
> Change SortDataRows into an External Sorter, it should sort in memory until it reach
configured size, then spill to disk. It should provide following interface:
> 1. addRow:  insert rows into the sorter.
> 2. getIterator: will return an iterator that iterates on sorted rows, the sorted row
could come from memory or files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message