carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravipesala <>
Subject [GitHub] carbondata pull request #1510: [WIP] Supported DataMap chooser and expressio...
Date Thu, 16 Nov 2017 13:27:26 GMT
GitHub user ravipesala opened a pull request:

    [WIP] Supported DataMap chooser and expression for supporting multiple datamaps in single

      This PR supports 3 features.
     1. Load datamaps from the DataMapSchema which are created through DDL.
     2. DataMap Chooser: It chooses the datamap out of available datamaps based on simple
logic. Like if there is filter condition on column1 then for supposing 2 datamaps(1. column1
2. column1+column2) are supporting this column then we choose the datamap which has fewer
columns that is the first datamap.
     3. Expression support: Based on the filter expressions we convert them to the possible
DataMap expressions and do apply expression on it. 
      For example, there are 2 datamaps available on table1 
      Datamap1 : column1
      Datamap2 : column2 
      Query: `select * from table1 where column1 ='a' and column2 = `b` `
      For the above query, we create datamap expression as `AndDataMapExpression(Datamap1,
DataMap2)`. So for the above query both the datamaps are included and the output of them will
be applied AND condition to improve the performance
     - [X] Any interfaces changed?
       Yes, DataMap interfaces are changed in order to support datamap expression.
     - [X] Any backward compatibility impacted?
     - [X] Document update required?
       Yes, dev guide need to be updated
     - [X] Testing done
           Tests are added
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella

You can merge this pull request into a Git repository by running:

    $ git pull datamap-fg-chooser1

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1510
commit c2b898468acdfb3d1738c34ce0557100700f6f80
Author: ravipesala <>
Date:   2017-11-15T14:18:40Z

    Added FG implementation and testcases

commit 3a405355874c051fd640cc3ad00049a1a73fd8c6
Author: ravipesala <>
Date:   2017-11-08T16:00:32Z

    Added DataMapChooser



View raw message