carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pawan Malwal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-1889) Block pruning not working for date type column
Date Wed, 13 Dec 2017 06:04:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pawan Malwal updated CARBONDATA-1889:
-------------------------------------
    Description: 
    spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, c4 bigint, c5
short, c6 date) stored by 'carbondata'""".stripMargin).show()

    spark.sql(s"""insert into test_dateType select 1,'1111111',111,1111111111,1111,'2017-12-12'""".stripMargin)
    spark.sql(s"""insert into test_dateType select 2,'2222222',222,2222222222,2222,'2018-11-11'""".stripMargin)
    spark.sql(s"""insert into test_dateType select 3,'3333333',333,3333333333,3333,'2019-10-10'""".stripMargin)

    spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' """.stripMargin).show(200,false)

Check for "Identified no.of.blocks" in logs

It shows :  Identified no.of.blocks: 3

Only 1 block should be selected but all the 3 blocks are getting selected

  was:
    spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, c4 bigint, c5
short, c6 date) stored by 'carbondata'""".stripMargin).show()

    spark.sql(s"""insert into test_dateType select 1,'1111111',111,1111111111,1111,'2017-12-12'""".stripMargin)
    spark.sql(s"""insert into test_dateType select 2,'2222222',222,2222222222,2222,'2018-11-11'""".stripMargin)
    spark.sql(s"""insert into test_dateType select 3,'3333333',333,3333333333,3333,'2019-10-10'""".stripMargin)

    spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' """.stripMargin).show(200,false)

Check for "Identified no.of.blocks" in logs

It shows :  Identified no.of.blocks: 3

Only 1 block should be selected


> Block pruning not working for date type column
> ----------------------------------------------
>
>                 Key: CARBONDATA-1889
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1889
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Pawan Malwal
>            Assignee: Pawan Malwal
>
>     spark.sql(s"""create table test_dateType(c1 int, c2 string, c3 smallint, c4 bigint,
c5 short, c6 date) stored by 'carbondata'""".stripMargin).show()
>     spark.sql(s"""insert into test_dateType select 1,'1111111',111,1111111111,1111,'2017-12-12'""".stripMargin)
>     spark.sql(s"""insert into test_dateType select 2,'2222222',222,2222222222,2222,'2018-11-11'""".stripMargin)
>     spark.sql(s"""insert into test_dateType select 3,'3333333',333,3333333333,3333,'2019-10-10'""".stripMargin)
>     spark.sql(s"""select * from test_dateType where c6 = '2018-11-11' """.stripMargin).show(200,false)
> Check for "Identified no.of.blocks" in logs
> It shows :  Identified no.of.blocks: 3
> Only 1 block should be selected but all the 3 blocks are getting selected



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message