carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ffpeng90 <>
Subject [GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto
Date Wed, 15 Mar 2017 02:53:14 GMT
Github user ffpeng90 commented on the issue:
        1. This version only suppport DML,  
            All tables for test are created by spark-sql(DML part), 
            and i submit queries to presto to get results.
            I only tested the "Select" Case , like where, group , sum , join.
        2.  I use APIs like createQueryPlan, resolveFilter from class "CarbonInputFormatUtil".

           To read carbon formatted table, i make the read process into several steps:
           a). load table metadata 
           b). get splits from table (pushing down filtering to filter datablocks of one segment
@CarbonTableReader.getInputSplits2 )
           c). parse records ( pushing down column projection and filtering into QueryModel
 @CarbondataRecordSetProvider.getRecordSet ) 
        3. As i described  in partC "parse records", I use QueryModel to get  decoded records.
           For lazy decoding,  I will keep on exploring a better solution.  Maybe we can get
inspiration from module presto-orc, presto-parquet.
    At 2017-03-15 09:11:19, "Jacky Li" <> wrote:
    Thanks for working on this. Can you describe what feature is added in term of:
    What SQL syntax is supported? DDL &DML?
    I think it uses CarbonInputFormat to read, so are you pushing down column projection and
filtering by setting the configuration in CarbonInputFormat?
    Is there any SQL optimization integration with Presto's optimizer? like leveraging carbon's
global dictionary to do lazy decode?
    You are receiving this because you authored the thread.
    Reply to this email directly, view it on GitHub, or mute the thread.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message