drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zelaine Fong (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-3881) Rowkey filter does not get pushed into Scan
Date Thu, 25 Feb 2016 19:33:18 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zelaine Fong updated DRILL-3881:
--------------------------------
    Fix Version/s:     (was: Future)
                   1.7.0

> Rowkey filter does not get pushed into Scan
> -------------------------------------------
>
>                 Key: DRILL-3881
>                 URL: https://issues.apache.org/jira/browse/DRILL-3881
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 1.2.0
>            Reporter: Khurram Faraaz
>            Assignee: Smidth Panchamia
>            Priority: Critical
>             Fix For: 1.7.0
>
>
> Rowkey filter does not get pushed down into Scan
> 4 node cluster CentOS
> Drill master commit ID: b9afcf8f
> case 1) Rowkey filter does not get pushed into Scan
> {code}
> 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB')
AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE ROW_KEY = CAST('3.0838087E38'
AS FLOAT);
> +------+------+
> | text | json |
> +------+------+
> | 00-00    Screen
> 00-01      Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
> 00-02        SelectionVectorRemover
> 00-03          Filter(condition=[=($0, CAST('3.0838087E38'):FLOAT NOT NULL)])
> 00-04            Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl,
startRow=null, stopRow=null, filter=null], columns=[`*`]]])
> {code}
> case 2) Rowkey filter does not get pushed into Scan
> {code}
> 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB')
AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB')
= CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%'
order by CONVERT_FROM(ROW_KEY,'FLOAT_OB') ASC;
> +------+------+
> | text | json |
> +------+------+
> | 00-00    Screen
> 00-01      Project(RK=[$0], EXPR$1=[$1])
> 00-02        SelectionVectorRemover
> 00-03          Sort(sort0=[$0], dir0=[ASC])
> 00-04            Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1,
'qual1'))])
> 00-05              SelectionVectorRemover
> 00-06                Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT
NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
> 00-07                  Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl,
startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)],
columns=[`*`]]])
> {code}
> Same as case (2) just that ASC is missing in order by clause.
> {code}
> 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB')
AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB')
= CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%'
order by CONVERT_FROM(ROW_KEY,'FLOAT_OB');
> +------+------+
> | text | json |
> +------+------+
> | 00-00    Screen
> 00-01      Project(RK=[$0], EXPR$1=[$1])
> 00-02        SelectionVectorRemover
> 00-03          Sort(sort0=[$0], dir0=[ASC])
> 00-04            Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1,
'qual1'))])
> 00-05              SelectionVectorRemover
> 00-06                Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT
NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
> 00-07                  Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl,
startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)],
columns=[`*`]]])
> {code}
> Snippet that creates and inserts data into HBase table.
> {code}
> public static void main(String args[]) throws IOException {
>         Configuration conf = HBaseConfiguration.create();
>         conf.set("hbase.zookeeper.property.clientPort","5181");
>         HBaseAdmin admin = new HBaseAdmin(conf);
>         if (admin.tableExists("flt_Tbl")) {
>             admin.disableTable("flt_Tbl");
>             admin.deleteTable("flt_Tbl");
>         }
>         HTableDescriptor tableDesc = new
>             HTableDescriptor(TableName.valueOf("flt_Tbl"));
>         tableDesc.addFamily(new HColumnDescriptor("colfam1"));
>         admin.createTable(tableDesc);
>         HTable table  = new HTable(conf, "flt_Tbl");
>         //for (float i = (float)0.5; i <= 100.00; i += 0.75) {
>         for (float i = (float)1.4E-45; i <= Float.MAX_VALUE; i += Float.MAX_VALUE
/ 64) {
>         byte[] bytes = new byte[5];
>         org.apache.hadoop.hbase.util.PositionedByteRange br =
>               new org.apache.hadoop.hbase.util.SimplePositionedByteRange(bytes, 0, 5);
>         org.apache.hadoop.hbase.util.OrderedBytes.encodeFloat32(br, i,
>               org.apache.hadoop.hbase.util.Order.DESCENDING);
>         Put p = new Put(bytes);
>         p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),String.format("value %05f",
i).getBytes());
>         table.put(p);
>         }
>         table.flushCommits();
>         table.close();
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message