hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Hanson (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-4946) Allow prevention of string column re-use for string functions that can set results by reference
Date Sat, 27 Jul 2013 00:01:48 GMT
Eric Hanson created HIVE-4946:
---------------------------------

             Summary: Allow prevention of string column re-use for string functions that can
set results by reference
                 Key: HIVE-4946
                 URL: https://issues.apache.org/jira/browse/HIVE-4946
             Project: Hive
          Issue Type: Sub-task
    Affects Versions: vectorization-branch
            Reporter: Eric Hanson
            Priority: Minor


In order to prevent a bug, I had to use BytesColumnVector.setVal instead of BytesColumnVector.setRef
when creating the output of all string functions. These include TRIM/LTRIM/RTRIM/SUBSTR, which
can be made faster if they can use the setRef method instead. That allows them to avoid the
cost of copying the data, by just setting a reference to a string (by setting the byte[] pointer,
start, and length). 

As a future performance enhancement, it would be desirable to be able to use setRef for these
functions instead of setVal. But to do that, it is necessary to be able to mark a BytesColumnVector
that is being referenced into so it is not reclaimed and re-used. So this would require a
design and implementation change to the output column manager used in VectorizationContext.

I'm marking this as "minor" priority because it will result in a small performance enhancement,
most likely. It can be deferred for a while.

The following is an example of a query that will exhibit a bug if setRef is used instead of
setVal in the implementation of trim and concat functions:

select l_shipmode, 
  rtrim(concat(l_shipmode,' '))  -- missing last char
  ,trim(concat(' ',l_shipmode))  -- requires this line for bug to show up
from lineitem_orc 
where l_orderkey = 1;



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message