Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Tue, 24 Oct 2017 22:04:00 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.13109801.1508188041000.50862.1508882640201@Atlassian.JIRA>
In-Reply-To: <JIRA.13109801.1508188041000@Atlassian.JIRA>
References: <JIRA.13109801.1508188041000@Atlassian.JIRA> <JIRA.13109801.1508188041706@jira-lw-us.apache.org>
Subject: [jira] [Commented] (DRILL-5879) Optimize "Like" operator
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 24 Oct 2017 22:04:05 -0000


    [ https://issues.apache.org/jira/browse/DRILL-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217782#comment-16217782 ] 

ASF GitHub Bot commented on DRILL-5879:
---------------------------------------

Github user sachouche commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1001#discussion_r146705325
  
    --- Diff: exec/java-exec/src/main/codegen/templates/CastFunctionsSrcVarLenTargetVarLen.java ---
    @@ -73,6 +73,9 @@ public void eval() {
         out.start =  in.start;
         if (charCount <= length.value || length.value == 0 ) {
           out.end = in.end;
    +      if (charCount == (out.end - out.start)) {
    +        out.asciiMode = org.apache.drill.exec.expr.holders.VarCharHolder.CHAR_MODE_IS_ASCII; // we can conclude this string is ASCII
    --- End diff --
    
    - As previously stated (when responding to Paul'd comment), the expression framework is able to use the same VarCharHolder input variable when it is shared amongst multiple expressions
    - If the original column was of type var-binary, then the expression framework will include a cast to var-char
    - The cast logic will also compute the string length
    - Using this information to deduce whether the string is pure ASCII or not
    - UTF-8 encoding uses 1 byte for ASCII and 2, 3, or 4 for other character sets
    - If the encoded length and character length are equal, then this means this is an ASCII string 


> Optimize "Like" operator
> ------------------------
>
>                 Key: DRILL-5879
>                 URL: https://issues.apache.org/jira/browse/DRILL-5879
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Relational Operators
>         Environment: * 
>            Reporter: salim achouche
>            Assignee: salim achouche
>            Priority: Minor
>             Fix For: 1.12.0
>
>
> Query: select <column-list> from <table> where colA like '%a%' or colA like '%xyz%';
> Improvement Opportunities
> # Avoid isAscii computation (full access of the input string) since we're dealing with the same column twice
> # Optimize the "contains" for-loop 
> Implementation Details
> 1)
> * Added a new integer variable "asciiMode" to the VarCharHolder class
> * The default value is -1 which indicates this info is not known
> * Otherwise this value will be set to either 1 or 0 based on the string being in ASCII mode or Unicode
> * The execution plan already shares the same VarCharHolder instance for all evaluations of the same column value
> * The asciiMode will be correctly set during the first LIKE evaluation and will be reused across other LIKE evaluations
> 2) 
> * The "Contains" LIKE operation is quite expensive as the code needs to access the input string to perform character based comparisons
> * Created 4 versions of the same for-loop to a) make the loop simpler to optimize (Vectorization) and b) minimize comparisons
> Benchmarks
> * Lineitem table 100GB
> * Query: select l_returnflag, count(*) from dfs.`<source>` where l_comment not like '%a%' or l_comment like '%the%' group by l_returnflag
> * Before changes: 33sec
> * After changes    : 27sec


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)