carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From simafengyun <...@git.apache.org>
Subject [GitHub] incubator-carbondata issue #627: CARBONDATA-748
Date Tue, 07 Mar 2017 10:10:32 GMT
Github user simafengyun commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/627
  
    
    
    You mentioned the below,
    >>>>>>>>>>>>>>>>>>>>>>>>>>
    But using this search in setFilterdIndexToBitSet is not always correct. Because here the
data may not be sorted always, Only for the first column of dimensions is sorted naturally
because of mdk order.
    >>>>>>>>>>>>>>>>>>>>>>>>>>
     but i don't think so. 
    the order you mentioned is the logical level order(MDK).
    As I know, for dimension column, it has physical order in chunk level.
    for dimension data which has dictionary encode, the dictionary data will sorted in blocklet
level and keep the order in chunk on the physical  disk.
    So after the one chunk dimension data read, it will keep the order, so I think it is fit
for the binary search.
    
    
    if I was wrong, please feel free to tell me, thanks 
    
    
    
    
    
    
    
    
    
    
    At 2017-03-07 17:44:28, "Ravindra Pesala" <notifications@github.com> wrote:
    
    
    @simafengyun Thanks for working on it.
    Range binary search is impressive. we can use same search in other places as well.
    But using this search in setFilterdIndexToBitSet is not always correct. Because here the
data may not be sorted always, Only for the first column of dimensions is sorted naturally
because of mdk order.
    And if the data is sorted explicitly(it means it has inverted index) then it goes to another
method setFilterdIndexToBitSetWithColumnIndex. So here we need extra checks to do binary search
in the method setFilterdIndexToBitSet , that is like whether the column is naturally sorted
or not.This information we may need get from store.
    Please use this Range binary search in 'setFilterdIndexToBitSetWithColumnIndex' as well.
    
    —
    You are receiving this because you were mentioned.
    Reply to this email directly, view it on GitHub, or mute the thread.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message