drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3881) Rowkey filter does not get pushed into Scan
Date Thu, 01 Oct 2015 18:02:27 GMT
Khurram Faraaz created DRILL-3881:
-------------------------------------

             Summary: Rowkey filter does not get pushed into Scan
                 Key: DRILL-3881
                 URL: https://issues.apache.org/jira/browse/DRILL-3881
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.2.0
            Reporter: Khurram Faraaz
            Assignee: Smidth Panchamia


Rowkey filter does not get pushed down into Scan

4 node cluster CentOS
Drill master commit ID: b9afcf8f

case 1) Rowkey filter does not get pushed into Scan

{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB')
AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE ROW_KEY = CAST('3.0838087E38'
AS FLOAT);
+------+------+
| text | json |
+------+------+
| 00-00    Screen
00-01      Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
00-02        SelectionVectorRemover
00-03          Filter(condition=[=($0, CAST('3.0838087E38'):FLOAT NOT NULL)])
00-04            Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl,
startRow=null, stopRow=null, filter=null], columns=[`*`]]])
{code}

case 2) Rowkey filter does not get pushed into Scan

{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB')
AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB')
= CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%'
order by CONVERT_FROM(ROW_KEY,'FLOAT_OB') ASC;
+------+------+
| text | json |
+------+------+
| 00-00    Screen
00-01      Project(RK=[$0], EXPR$1=[$1])
00-02        SelectionVectorRemover
00-03          Sort(sort0=[$0], dir0=[ASC])
00-04            Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1,
'qual1'))])
00-05              SelectionVectorRemover
00-06                Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT
NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
00-07                  Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl,
startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)],
columns=[`*`]]])
{code}

Same as case (2) just that ASC is missing in order by clause.
{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(ROW_KEY,'FLOAT_OB')
AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB')
= CAST('3.0838087E38' AS FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%'
order by CONVERT_FROM(ROW_KEY,'FLOAT_OB');
+------+------+
| text | json |
+------+------+
| 00-00    Screen
00-01      Project(RK=[$0], EXPR$1=[$1])
00-02        SelectionVectorRemover
00-03          Sort(sort0=[$0], dir0=[ASC])
00-04            Project(RK=[CONVERT_FROMFLOAT_OB($0)], EXPR$1=[CONVERT_FROMUTF8(ITEM($1,
'qual1'))])
00-05              SelectionVectorRemover
00-06                Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), CAST('3.0838087E38'):FLOAT
NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 'UTF8'), '%30838087473969088%'))])
00-07                  Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl,
startRow=, stopRow=, filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, ^.*\x5CQ30838087473969088\x5CE.*$)],
columns=[`*`]]])
{code}

Snippet that creates and inserts data into HBase table.

{code}
public static void main(String args[]) throws IOException {
        Configuration conf = HBaseConfiguration.create();
        conf.set("hbase.zookeeper.property.clientPort","5181");
        HBaseAdmin admin = new HBaseAdmin(conf);
        if (admin.tableExists("flt_Tbl")) {
            admin.disableTable("flt_Tbl");
            admin.deleteTable("flt_Tbl");
        }

        HTableDescriptor tableDesc = new
            HTableDescriptor(TableName.valueOf("flt_Tbl"));

        tableDesc.addFamily(new HColumnDescriptor("colfam1"));
        admin.createTable(tableDesc);

        HTable table  = new HTable(conf, "flt_Tbl");

        //for (float i = (float)0.5; i <= 100.00; i += 0.75) {
        for (float i = (float)1.4E-45; i <= Float.MAX_VALUE; i += Float.MAX_VALUE / 64)
{
        byte[] bytes = new byte[5];
        org.apache.hadoop.hbase.util.PositionedByteRange br =
              new org.apache.hadoop.hbase.util.SimplePositionedByteRange(bytes, 0, 5);
        org.apache.hadoop.hbase.util.OrderedBytes.encodeFloat32(br, i,
              org.apache.hadoop.hbase.util.Order.DESCENDING);

        Put p = new Put(bytes);
        p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),String.format("value %05f",
i).getBytes());
        table.put(p);
        }
        table.flushCommits();
        table.close();
    }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message