hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2327) Optimize REGEX UDFs with constant parameter information
Date Fri, 15 May 2015 02:50:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544825#comment-14544825
] 

Ashutosh Chauhan commented on HIVE-2327:
----------------------------------------

No there is no standard approach. Your test sounds good. I will modify the query to have a
map-only query, so that other overheads don't come in play, something like:
{code}
select t1 from t where t.t1 regexp '.*abc.*';
{code}

> Optimize REGEX UDFs with constant parameter information
> -------------------------------------------------------
>
>                 Key: HIVE-2327
>                 URL: https://issues.apache.org/jira/browse/HIVE-2327
>             Project: Hive
>          Issue Type: Improvement
>          Components: UDF
>            Reporter: Adam Kramer
>            Assignee: Alexander Pivovarov
>         Attachments: HIVE-2327.01.patch, HIVE-2327.2.patch
>
>
> There are a lot of UDFs which would show major performance differences if one assumes
that some of its arguments are constant.
> Consider, for example, any UDF that takes a regular expression as input: This can be
complied once (fast) if it's a constant, or once per row (wicked slow) if it's not a constant.
> Or, consider any UDF that reads from a file and/or takes a filename as input; it would
have to re-read the whole file if the filename changes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message