carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ravipesala <>
Subject [GitHub] carbondata pull request #1179: [WIP] Added the blocklet info to index file a...
Date Mon, 17 Jul 2017 15:33:01 GMT
GitHub user ravipesala opened a pull request:

    [WIP] Added the blocklet info to index file and make the datamap distributable with job

    In this PR following tasks are completed.
    1. Added the blocklet info to the carbonindex file, so datamap not required to read each
carbondata file footer to the blocklet information. This makes the datamap loading faster.
    2. Made the data map distributable and added the spark job. So datamap pruning could happen
distributable and pruned blocklet list would be sent to driver.
    This PR cannot compile as carbondata format changes are present.

You can merge this pull request into a Git repository by running:

    $ git pull datamap

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1179


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message