carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravindra Pesala (JIRA)" <>
Subject [jira] [Assigned] (CARBONDATA-1230) Datamap framework for Carbondata to leverage indexing
Date Tue, 27 Jun 2017 06:30:01 GMT


Ravindra Pesala reassigned CARBONDATA-1230:

    Assignee: Ravindra Pesala

> Datamap framework for Carbondata to leverage indexing
> -----------------------------------------------------
>                 Key: CARBONDATA-1230
>                 URL:
>             Project: CarbonData
>          Issue Type: New Feature
>            Reporter: Ravindra Pesala
>            Assignee: Ravindra Pesala
> Datamap should be single point interface for indexing and pruning. 
> It could be two types
> # 1. Coarse grained datamap.
> # 2 Fine grained datamap.
> h3. Coarse grained datamap
> These datamaps contains the information of blocklets. so it can prune till blocklet level.
It could be loaded on driver side or executor side depends on size of datamap.
> Default implementation for this type is BlockletDataMap. It contains all necessary information
 of blocklet with stats like startkey, endkey and max and min value. Using this information
all filter queries would be pruned by datamap.
> h3. Fine grained datamap
> These datamap contains information up to page and row level. It is stored executor side
and used as part of filtering to speed up the queries.

This message was sent by Atlassian JIRA

View raw message